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ABSTRACT 

This study, concerning the conceptual and 
econometric problems involved in estimating educational production 
functions, focuses on the following topics; 1) the meaning of an 
educational production function estimated from cross-section data; 2) 
the measurement of the output of schools; 3) the problem of measuring 
the initial endowment of students upon entering school; 4) the 
measurable dimensions of learning environment, both school and home; 
5) the shortcomings of the Project Talent five-year follow-up data; 
and 6) estimated educational production functions, using Project 
Talent data as well as data from the Equal Educational Opportunity 
Survey of the Office of Education. Some major findings are: a) the 
relationships are consistent with the conceptual model 
in this report; b) teacher quality appears to be an 
determinant of scholastic success; and c) the production 
explain a very small percentage of the variance of 
scholastic achievement, even using the full range of social class and 
school input variables. (Author/LS) 
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This study concerns the conceptual and econometric problems 



involved in estimating educational production functions. Atten- 



tion is given to the following topics •' First, the meaning of 
an educational production function estimated from croSs-section 



data is discussed. Attention is given to the problem of simul- 



taneity, and to the difficulties arising from the absence of 

i • , . • 

the usual maximizing behavioral assumptions which ordinarily 



underlie production function estimates. Second, I deal with 



the measurement of the output of schools, in this study, I 

o* . 

concentrate on achievement scores as a measure of output, al- 

j 

though there is some attention given to economic measures, such 



as post-school earnings. Third, I discuss the problem of mea- 



suring the initial endowment of students upon entering school. 



A method of dealing with the normal mis-specification of educa- 
tional production functions arising from this source is developed 



and implemented. Fourth, the measurable dimensions of the 



learning environment, both school and home, are discussed, A 



model of the learning environment is developed, based on the 



findings of sociological and psychological research. Fifth, 



the shortcomings of the Project Talent five-year follow-up data 



are discussed. Particularly important is the magnitude of non- 



response to the follow-up and non-response on particular items 



by those included in the follow-up. Sixth, educational prpduction 
functions are estimated using Project Talent data, as we.li as 
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data from the Equal Educational Opportunity Survey of the Office 
of Education. The following findings are particularly important : 



a) the estimated relationships are consistent with the con- 



ceptual model developed in this report; bj teacher quality 



appears to be an important determinant of scholastic success; 



c) some other dimensions of the school environment appear to be 
important, although the relationships are somewhat inconsistent; 



and d) the production functions explain a very small percentage 



of the variance of scholastic achievement, even using the full 



range of social class and school input variables. 
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I . PREFACE 

The following is the final report on the first stage of a 
research project on educational production functions supported 
by the U.S. Office of Education under grant number OEC 1-7-0C0451-2 

The first stage has been devoted to an exploration of the 

i 

conceptual and econometric problems involved in the construction 
of ’ educational production functions, and the estimation of some 
preliminary functions in which the school output is measured by 
scholastic achievement. It was originally planned that the data 
used in the first stage would be drawn entirely from Project 
Talent However, un&reseen delays in acquiring the necessary 
tapes have ' resulted in a somewhat restricted use of Talent data 
supplemented in part by data from the Office of Education's 
Equality of Educational Opportunity Survey, 

The second stage is devoted to the economics of educational 
production functions. Here the emphasis will be on the relation- 
ship between school inputs, social class, and post school earnings, 
occupational attainment, and employment status. Some exploratory 
analysis in this area was included in stage one. The preliminary 

I 

results are outlined in the postscript to this report. The data 
used in this economic analysis are from Project Talent, 

^During the period of this contract (11/1/66 through 12/31/68), 

I have received support from other sources for study in these 
problems. This report constitutes a summary of my progress, 

TO identify particular findings or concepts with specific sources 

im r\P n*- 1 and i mrinssible . 
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I would like to thank a number of people for assistance. 
Pamela King, Alicp Crampin, Mathew Lambrinides, and Keith 
Clemens served capably as research assistants, i received useful 
advice from Zvi Griliches, Susan Contratto, Christopher Jencks, 
Lester Thurow, Stephan Michelson , Henry Levin, Arthur MacEwan, 
Thomas Weiskopf, and Christopher Sims. Terry Rothra and Deanna 

i 

Lee typed the manuscript 

Researchers using Project Talent data are asked to include 
the following statement in their report: "This investigation 

utilized the Project Talent Data Bank, a cooperative effort of 
the U.S. Office of Education, the University of Pittsburgh, and 
the American Institutes for Research. The design and interpre- 
tation of the research reported herein, however, are solely the 
responsibility of the author." 
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An educational production function relates school 



and student inputs to a measure of school output, ’^epresen- 
tc!i ion of Ute .educational production process in this form is 
' i»u ticular |interest in the descriptive study of human 
v^apital formation as well as in norn^ative investigations of 
t lie optimal allocation of resources in the educational sector. 

. “ I 

If schooling has any unique' effect on' "labor produc- 
tivity or earnings, this effect should be traceable to the 
ti<-.velopmc.nL of; cognitive skills and attitudes as a consequence 

of school attendance. Further, we may be able to relate the 

I ■ ■ 

development of| productive personal attributes • to school- 

f;ol ic los •conctirniug- the allocaticni of scarce educational 

la'sourt'es . A pfiKluction function relating school inputs to 



I he dove lopnuiui of an i nd i v idua 1 * productive capacity wouhi 
qiv#’' MS a much better idea why the bet ter-educated i-arn more. 

• i 

Moreover, by ipvestigat ing differences in production 'funcc. icns 
tor different racial and social class groups, as well as 






/':fferences in educational inputs among these a roups, we mav 



bettei undiM Stand one important aspect of the determinat ion 



• q- ihc' disl.ritnition of personal earnings. 



In the determination of school policy, and in long* 



run educational planning, knowledge of the educationall pro- 



duction function is essential to the achievement of efficient 



.’.•'source allocation. This is true, of course, regardless of 
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Wli. ( lioi- tlK- dt'oision unit: is ptu'suing the objective of growth, 
equolity, or any combination of these and other goals. 

Without an estimate of the technology of education, the rela- 

i 

lion between the opportunity costs of particular VKilicies and 
their expected benefits must be little more than guesses. 

An educational production function is defined as follows: 



( 1 ) 

where 






/ • • • 



' *m' *n' • • •' • • •. X_) 



w 



I X 
{ 



A - Lome measure of school outpul -- for example, a 
score on a scholastic achievement battery; 

^l'***' \ “ variables measuring the school environment. The 

Variables here would typically include the amount 
and quality of teaching services, the physical 
facilities of the school, the length of time that 

i 

the student is exposed to these inputs; 

Xn^*../ = variables representing environmental influences 

<!>n learning outside the school — e.g., the parents' 

' * - ’ • t 

educational attainment; 

4 ♦ 

*»'•••' *2 “ variables representing the initial level of learning 
-- attained by the student prior to entry into the 

type of schooling in question. 

- We are interested in gaining estimates of the structural 

parameters of the function, f. it will be seen below that we 

I 

cannot estimate the above equation in the form presented, although 

I 

some progress pan be made with a slightly modified version. 
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Tho (lata at our disposal are ordinarily based on 



cl 



cross-aoclion of students. Altiiough T will dwell at nome 



len.jth on the deficiencies of our data, the information avail- 



able tor the estimation of educational production functions is. 



1 n mil any superior to that underlying production 

IfuncL ion estimates in the economy. The crucial deficiency, it 



will be seen, is not so much in the absence of data as in the 



absence of a theory of the learning process which will guide us 



in the process of estimation. The engineering processes used in 



the production of physical commodities are reasonably well under- 



stood. They suggest appropriate specifications of the production 

1 • ’ , 1 ' 

function, as w^ll as some ^ priori limits on what are regarded as 

plausible estiijiates. In the estimation of educational production 
fiinctions, the psychologist replaces the engineer or agronomist 



as the source of technical information on the production process. 



Despite some fruitful developments in learning theory, we are 



left without much guidance for the underlying technical pro- 



cesses involved. 



None|:heless, to preview some of our results, it will 



1 ^.# 



e siu.-n thut a reasonable a pri or i model of the production of 



Sv.-hol .1st ic nehievement can be specified on the basis of 

i 

rxisting theory. Moreover, preliminary estimates of this 



T 



t'uact. ion are encouraging. 



e 

ERIC 






4 



- 6 - 






Attempts to measure the relationship between school inputs 

and outputs have occipied the attention of a number of educational 

researchers over the last half-century, yet the estimation of 

the structural parameters of a production function similar to 

2 

(1) is relatively new. The results of the studies completed to 
date are difficult to summarize, in part because of the large 
variety of measurements used, and in part because of the diversity 
of findings. In any case, the purpose of this paper is not primar 
lly to present empirical estimates of production functions, but 



2 

As I will refer to the methods and results of studies in the 
course of the paper, I will briefly review them now, (All works 
referred to in this report appear in the bibliography.) 

Herbert Kies ling used data generated by the Quality Measurement 
Project of New York State to estimate school production functions 
for various communities in New York, Martin Katzman estimated 
production functions for a variety of school outputs of elementary 
schools in Boston. As a part of the study which gave rise to the 
report of the Central Advisory Council for Education (the Plowden 
Report) in England, G, F. Peaker estimated a series of production 
functions for British elementary education. Thomas Pox and John 
Holland and Jesse Burkhead have estimated production functions 
for a wide range of school outputs for Atlanta and Chicago, as 
reported in Burkhead. I have not included in this list the study 
of Finis Welch, as he relies on highly aggregated inputs and his 
estimates can only be identified as educational production 
functions by some stretch of the imagination. Eric Hanushek and 
David Armor have used U.S. data on the sixth grade to estimate 
production functions for elementary education. The international 
Project for the Evaluation of Educational Achievement, under the 
direction of Torsten Hhsen, has estimated similar functions for 
the determination of mathematics achievement in a sample of 12 
countries. A considerable amount of additional work is now in 
progress. 




rather to explore some of the conceptual and econometric problems 
involved in this type of estimation. Nonetheless, the results of 
some of these studies, as well as my own results, will be intro- 
duced as illustrations. 

Part III will include a discussion of the behavioral assumptions 

i ' • . ■ 

underlying the usual production function estimation, and the 
particular difficulties encountered when the concepts are applied 

I . . . ■ . . 

to schools. Parts IV and V are devoted to the measurement of 
school outputs and student inputs of the production process. The 
measurement and interpretation of school inputs is discussed in 
part VI; and in Part VII the statistical properties of the project 
Talent data are surveyed, in Part VIII some results based on 
Project Talent data are presented. These results are compared 
with estimates based on EEOS data in Part IX. Problems of sped- 
fication bias are discussed in Part X. Part XI is a brief con- 



clusion 






R; ^ 

I \ III. ESTIMATING A PRODUCTION MODEL FOR SCHOOLS 
] The striking characteristic about the production process in 

schools is the degree to which it appears to be complex, unsystem- 
atic, or just plain not understandable, in a statistical investi- 
gation using non -experimental data, the most we can expect is 
j discovery of some of the relationships among measured dimensions 

I " ' ' ' 

of the process based on the particular configuration of data in 

* . 

our sample. We are thus limited both by the preconceptions of 
the researchers responsible for the selection of the sample and 
the available data as well as the patterns of variation which 

school decision-making processes have brought about in the sample 

. 

of schools chosen. To use the apt analogy of Marshak and Andrews, 

t 

we are not in the position of the agronomist who seeks to under- 
stand production relations in agriculture with a mind to making 
agriculture more productive. He can experiment, varying his 
factor inputs systematically and in any desired combination, and 
thus, under ideal conditions, predict the likely consequences of 
changes in factor inputs on productivity. Nor are we in the 
position of the meterologist who relies on non -experimental data, 
but seeks only to predict normal behavior rather than to effect 
events. We have the worst of these worlds, for we seek to affect 
the pattern of educational output by altering school inputs, and 
yet our data are generated entirely by systems of decision- 
making and student responses entirely beyond our control. Thus 
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Thus we are faced with the usual problem of simultaneous equa- 
tion bias which has plagued the estimation of production 
functions at the firm level any single equation approach to 

i Iv: estimation of (2) will yield inconsistent estimates of 
the structural parameters 

.1 

■ One possible way out of this difficulty arises from 
the basic implausibility of the above behavioral model. It 
•nay be that school administrators do not select school inputs 



\s' if -they were maximizing any well defined function of school 
outputs. This seems a reasonable assumption, given that school 

administratior( know very little about the underlying technology 



> . ' ? 

and are subject to a wide variety of political and legal 



constraints 



In this case we can take the j as exogenous for the purposes of 

I I 

estimation. i 

■ * ■ . -1 

■ .. ,, . ! 

Rejection of an optimizing decision model for school admin- 

i 

istrators relieves us of at least one simultaneity problem (there 

■ 1 

\ . 

will be others), but it deprives us of the usual interpretation of 

I 

the estimated parameters of (2) as a production function. We 
ordinarily reserve for this concept a relation which indicates 

i 

the maximum output consistent with a given set of inputs. Yet 
if school administrators conform to no systematic optimizing 

I 

behavioral model, then the observations on which our estimates 
are based are not generally technically efficient. Thus we arrive 



4 

See Marshak and Andrews, and Werlove for a discussion. 
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our ability to calculate the consequences of departures from 
existing ways of producing education is very limited indeed.^ 



But the limited variation in the configuration of 
inputs of our sample of schools is just the beginning of the 
dif ficulty^. Postponing the discussion of the precise func- 
tioniil form to be used, assume for the moment that we seek 
to ci'stimate the production function (1) in the form 



( 2 ) 

where 



^i “ ^0 ^3^1i ^2^2i + • • • + ^z^zi ^i 



Aj^ = the achievement score (or other output measure) 
for the i^ student 

fQ,...,f^ = the parameters of the production function to be 

estimated 

Xj£ = the amount of input j (including measures of 
home environment ) devoted to observation of 
i*s education, j = l...z 
Uj^ 1= the disturbance term 

i . 

Yet we may expect that the school inputs are endogenous 
to some system, for example, a system of equations based on the 
school administrators social welfare function, the educational 
production funcbion(s), and an educational budget constraint. 

- I 

3 

A considerable amount of educational research has used experi- 
mental techniques. See, for example. Gray and Klaus, and Kirk. 

I These methods hold out some promise for empirical determination 

^ of the educational production function. 
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s' * 

t ^ 

at some sort of average production function. Only if the ab- 
I solute degree of inefficiency is uncorrelated with the level of 
factor inputs (which seems unlikely) will the estimates f^ from 
(2) represent unbiased estimates of the true underlying produc- 

5 ' ■ 

tion relation. 

While the determination of school inputs can perhaps 
plausibly be regarded as exogenous to our system, one set of 
inputs most certainly must be taken as endogenous — student 
attitudes toward themselves and toward learning. These are both 
important determinants of achievement and a consequence of the 
students' past and present achievement levels, as well as other 
influences, in this case simultaneity seems unavoidable. 

1 Estimates based on (1), including student attitudes as explanatory 
variables, will in general be correlated with the disturbance 
term. Our solution is to estimate (2), an equation in which 
attitudes are excluded, the explanatory variables being confined 

I ' . 

to those which are exogenous. This reduced form equation incorpor 

j 

ates the effects of attitudes indirectly as they are related to 
the set of exogenous variables. Unless we are interested in 
increasing scholastic achievement by directly affecting student 

i ^ 

5 ' 

Of course, the constant term will be biased downward. If we 

had a number of different observations on inputs for the same 

school, we could use school dtammy variables to eliminate this 

"management bias . " See Massell and Hoch. 

















attitudes, little is lost by excluding the attitude variables 
from the equation,^ 



The dirth of knowledge concerning the underlying learning 
relationships makes a priori specification of a functional form 
for the estimation of educational production relations particu- 
larly difficult. The notion of diminishing marginal product is 




['■ 



(■ 

■ 



I': 







an appealing one, although certainly not well established in the 
field of education, prom this standpoint a function linear in 
the logarithms of the variables would seem somewhat superior. 

The possibility of positive interactions between inputs also 
recommends this form. Nonetheless, the restrictions of the 
Cobb-Douglas function are severe — particularly important to my 
mind is the fact that the cross derivatives among any pair of 
inputs, each of which is positively related to output, must also 
be positive. This would require, for example, that increases in 
the quality of teachers are more effective among the children 
of well-educated parents. For reasons of simplicity, in the 
work below i will use the linear additive form presented in 
(2) above. ^ 

Not all children learn the same way or the same things. 
Lesser and Stodolsky, for example, found dramatic differences 
in the patterns of scholastic proficiency on four different 
learning dimensions among Chinese, Jews, Negroes, and Puerto 

Nonetheless, in my results I present both the reduced form and 
the (biased) estimates of the structural equation itself. See 
Part X, 









.1 

j 




"^Hanushek found that the logarithmic form gave slightly better J 

significance for the estimates of the parameters of his nroduc- 

I ERIC 
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Ricans. When we find consistent differences in patterns of 
response to school inputs, we have good grounds for grouping 
our students according to these systematic patterns and estimating 
a number of different technologies. Although I know. of no work 
presenting systematic statistical tests of the hypothesis that 
educational production functions estimated from sub-populations 
were drawn from the same underlying population, casual inspection 
of the results of Hanushek, Kiesling, and my own work strongly 
suggest that it is useful to think in terms of distinct educational 
production technologies, at least for black and white and rich 
and poor students, separately,® 

If we may take a lesson from the study of economic growth, 
we should anticipate that the major changes in productivity of 
school resources will come from changes in production functions, 
including changes in relations between home background and achieve- 
ment, as well as the more conventional input-output relations, if 
this is our goal, we should seek to identify 'best practice' 

schools and develop a quantitative explanation of their superior 
technique. 

7cont. . 

tion functions. 

8 ' 

In my results below i have estimated functions for black 12th 

grade students separately, sometimes with a regional stratifi- 
cation. 
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IV, SCHOOL OUTPUTS 

We are interested in the economic consequences of 
schooling. Thus our output measures ideally should concern 
economic and social behavior following the termination of 
I schooling. Characteristically, we are forced to use indices 

r of student 'achievement* based on. tests administered while 
I the youth is still enrolled. 'rhosc achieveoient scores must 

be considered cither proxies for, or perhaps influences on, 

^ post-school economic behavior., Scholastic achievement is 

f , . pres imably not valued per se , but only as an intermediate 

|: input into other valued measures of performance. Thus al- 

j tlioiMh we will here use achievement. A, as* the output measure, 

I oiu: rationale for doing tlii.s is a social welfare function, 

[ iiiany of whose arguments are themselves functions of scholas- 

I tic achievepient . 

Although the evidence of a relationship between scholastic 
achievement and earnings is not well established, we proceed on 
the assumption that scholastic achievement has economic conse- 
I quences^ at least for some major groups of workers.^ 

j ^See Hansen, Scanlon and Weisbrod; Duncan, and part XII of this 
report for some evidence on this question. 
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Scholastic achievcmonL , of course, is not a single 



dimension of school output, Tjitcrally hundreds of instrumeiits 
hcive been devised to measure achievement in school. And 
achievement as ordinarily defined on these tests is but one 
i'jispect of the consequences of schooling 'on the growth of 
cogni tive , skills and personality. In. addition to the effect 
of achievement on economic performance in the post-school 
years, we may be interested' in the effects of schooling directly 
on an ind ividual ' s . self confidence, self-concept, or his sense 
of control over his environment. Evidence of zero order correla 
tions among individual test scores, some of which are presented 
in Table 1, suggests that the relations ampiig at least some of 
these measures are rather weak. 

Thus the output of schools is multidimensional 
with a vengance, and to complicate matters, there arc no 
convenient sets of 'prices' with which to aggregate the 
output, of course, few problems would arise if we found that 
the technologies for the production of each dimension of the 
output were roughly similar. This, however, does not seem 
to be the case. Estimates of the reduced form equation (2) 
ill which the dependent variable is a measure of scholastic 
achievement, differ considerably from estimates in which 
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Table 1 

Zero-Order Correlations Among Measures of School Outputs . 

Twelfth Grade Boys, 

U.S. 







2 


3 


4 


5 


6 


1 . 


Information Total 


.23 


.65 


.76 


■ .54 


.19 


2 . 


Self-Confidence 


* 


.17 


. 1 ? 


.09 


.11 


3 . 


English Total 






.67 


.46 


.26 


4 . 


Mathematics Total 








i 57 


.20 


5 . 

6 . 


Abstract Reasoning 
Clerical Checking 






, ' 




.19 



The test scores are described in Project Talent , 

Flanagan (1964), 



o 

ERIC 



- 17 - 



an index of the student's sense of control over his environment 
is the dependent variable. Production function estimates for 
different types of scholastic achievement differ also (see part 
VIII), 



Apparently we require not just one production 
function, but many, which, along with given resource endow- 
m«^5nts and budget constraints, could determine a production 
posnibility set for the school. The production possibility 
set, along with a social welfare function indicating the rel- 
ative importance of the various dimensions of school output, 
wen Id then form the analytical basis for resource allocation 
in the school. 



the purposes of policy making, we are particularly 
interested in the structural parameters of the production func- 
tion ^1) , for under ideal conditions they may bo interpreted 
IS the marginal products of the inputs in question, that is. 

Ml ^ ^^j ” may use this information to move in the 

direction of optimal input proportions as defined by the con- 



tJit ions 



11 



^^The , social welfare function would presumably reflect a combina- 
tion of societal, parental and child interests. 



11 



Of course, we are here accounting for only one output. 
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(3) 



3A/3Xj t. 



for all pairsy j, k 



3A/3X,^ fk Pk 



However, difficulties arise when we seek to compare the marginal 
products of the same input for two different groups of students. 
We find, for example, that the estimate of the structural param- 
eter relating to the verbal ability of teachers as an input into 
an achievement production function is considerably greater for 
black twelfth graders in the U.S. than for whites. Can we infer 
from this that verbally adroit teachers ought to be shifted from 
white to black districts? 



The output measure is ordinal; there is no zero point and 



no well defined unit of measurement for achievement. 



12 



Thus , 



while the marginal rate of substitution in production — repre- 
sented in the additive linear form by the ratio of regression 
coefficients of any two input factors — is still a valid analyt- 
ical concept; the absolute magnitude of the marginal product is 
not. Among students scoring at very different parts of the 
scale of measurement, equal units of increase in scores are not 
comparable; for example, it may be "easier" to make gains at the 
lower end of the scale than at the upper end due to a so-called 
Iceiling effect.' We really need to know the relationship 



12 



At least one writer has constructed a cardinal/ index of 
achievement based on the size of vocabulary (Bloom, pages 103- 
104 > Whether words known is linearly related to anything impor- 
tant is not known. For the concept of a zero point, see Thur stone. 
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between our output measure. A, and measures of directly desired 
performance, such as earnings. 

Although there is some evidence of a linear relation 
between achievement and earnings, it is certainly not sufficient 
to justify much confidence in a cardinal interpretation of 
measures of school learning,. 

A further problem remains. Our output indices are subject 

to some error — that is, test score = "true measure" + error, 

and, consequently, var(test score) = var(true measure) + var 

(error) . We have no idea of the validity of the test — that 

is, its correlation with a hypothetical true measure. But some 

idea of the magnitude of the error may be gained from estimates 

of the reliability of the tests. The reliability of our tests 

13 

is in the neighborhood of .9; Taking this as an upper estimate 
of the validity, at least 19 per cent of the variance of the test 

scores is due to test errors. Assuming that the errors in test 
measurement are uncorrelated with our explanatory variables, 
even if our explanatory variables predict the true measure with 
perfect accuracy, a validity of .9 imposes an absolute maximum 
proportion of variance explained by our equations of .81. it 
^e seen below that the actual R^'s are considerably lower. 



'^Although there are various ways of measuring test reli- 
ability, we may convey the essential meaning as the zero order 
correlation between scores on the- odd and even number questions 
of the Scune test or the zero order correlation between two 
versions of the test given to the same individual at roughly 
the scune time. 



V. MEASURING THE STUDENT INPUT 

An achievement score must be considered a measure of gross 
output. Our goal is to estimate the relationship between school 
Inputs and net output, or value added. For this we need a measure 
of the raw material inputs, i.e., student ability, or, alterna- 
tively, the level, of learning upon entry to the school in question 

The problem is that all measures of relevant student 
ability* depend heavily on previous learning, and are hardly 
distinguishable from measures designed explicitly to test 
scholastic achievement. Intelligence, as measured by the 
standard I.Q. instruments, is a developmental concept for 
measuring general learning Moreover, most I.Q. tests depend 
heavily on verbal facility, which is probably a good reflection 

I I . • ‘ 

of general school learning, and which apparently develops 
I similarly in all children.^^ Evidence that 'abilities* measured 

t ^ ■ 

0 

in IQ tests are in large measure a product of the educational 

♦ 

I 

‘ environment is suggested by Table 2, based on a study of identical 

I 

twins who were separated prior to age three. Over 60 percent of 

1 ... 

the variance of differences in IQ can be explained by differences 

I 

i ' . 

in the educational environment. The physical and social environ- 

I 

( . , 

ment together explain less than a third of the variance of the IQ 

! ' i 

i 

differences. There is substantial further evidence on the lack 

^^See Hunt. 
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Bloom, pp. 71 and 104. 
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Table 2 



The Effect of Environmental Differences on I.Q. 
Differences of Identical Twins Reared Apart^ 



V* 



a) Data from Newman, Freema n , and Holzinger (1937). 



Environmental 

Difference 




Effect^ 


t statistic 




Educational 




.66 


1.2 




Social 




.25 


' ■ / 

1.6 




Physical 




.19 


1.3 




ii^ 




.70 






d.f. 
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bl _.„ Normalized Regression Coefficient of the Environmental 
Difference in an equation predicting I.Q. differences. 
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■v>: iiKlj-pendence between measures of ability and school learning 
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If 'ability* is not an operational concept in this context. 



flow do we intend to interpret the raw material input of the 



schooling production process? As we are interested in measuring 



^school learning, it would seem reasonable to use tests of learning 



administered at grade one as a measure of raw input. Beacuse these 



first grade tests clearly measure the combined effects of genetic 



ability and environmental influences prior to age six, they are 



exactly what we need. Thus, our basic equation is: 



(4) 



A^2 ** ^ ^^1 ^ * . * ■ * t f ^ 1 ^ * 



where subscripts on the achievement variable refer to the grade 
at which the test is taken. In order to estimate a function of 



this type, we need individual test scores for students at two 



different levels of schooling. While some data of this type 

17 



i/S currently available, and more is on the way, we are 
generally forced to rely on cross-sections. 



If (4) is the correctly specified relation, and we are 
forced to work with data which do not include the first grade 
scores (A^^) , we may be able to estimate the unbiased regression 
coefficients of (4) if we have independent evidence on b^ 
the regression coefficient of A^^ in equation (4), as well as the 
estimated equations: 

(5) 



.12 






( 6 ) 



A^ = . . .. X^) 



The unbiased estimates of the regression coefficients of (4) 



17 



from Proiect Talent, for example 
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are then 



(7) 



b* 



^12 

; " ^l,12^i 



where 



-^12 *^1 

bj^ f' jare the estimated regreiission coefficients of 



in equations (5) and (6), respectively. 



This approach is equivalent to Theil's method 

18 



of estimating 



the bias due to specification error. 



I have assumed that the relationship between first grade 
and twelfth grade scores is such that a student scoring one 
standard deviation above the mean at grade one will, cetei^is 
paribus , score ,5 standard deviations above the mean at grade 
12, Thus, 



( 8 ) 

where 






1,12 



a 2^2 “ the standard deviation of achievement 
scores at grades 1 and 12, respectively. 

This figure is somewhat arbitrary. It is based on two sets of 
data. First, longitudinal studies of scholastic achievement 
scores suggest a simple correlation between early and late 

I 

scores in the neighborhood of ,6 to ,9, Most of the studies 

cover substantially less than twelve years, so we may suspect 

/ 

that the simple correlation of scores at grade one and twelve 



18. 



Theil (]357) Our method is based on the assumption that 
the function, f> accurately represents the relationship between 
each X. and first grade scores which prevailed at the time of 
their school entry, and that the vector X., . . ., X is the 
same for a given student at grades one ana twelve. ^ 
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would be somewhat lower. Moreover, the simple correlation is not 

* 

the appropriate evidence, as we seek an estimate of the partial 
effects of differences in on ^ 22 * To the extent that students 

who initially score high on tests are exposed to a better learning 
environment, the size of the above reported correlations exaggerate 
the normalized partial relationship between initial endowments 




i 

: 



I 

i: 



and later scholastic achievement. 

The second set of data shows that group scores of students 
classed by socio-economic categories show roughly constant 
.patterns over the years of school. Groups who begin school a 
standard deviation below the mean end up twelve years later in 
roughly the same relative position. Given the observed differ- 
ences in the qualtity of the learning environments of these 
various groups, we may infer that the partial (normalized) rela- 
tionship between initial scores and 12th grade scores is less than 
unity. The choice of .5 is maybe too low, and reflects a desire 
not to overcorrect for specification bias and thus underestimate 
the importance of social class and home environment in the learn- 
ing process. 



^^Based on 41 longitudinal achievement score correlations reported 
in Bloom, pages 106-109. The correlations for more widely separ- 
ated years occupy the lower end of this range. 



^^Coleman, et al,, ch. 3. 










All of the achievement measures are subject to error. At grade 
one the reliability of the achievement score used (verbal ability) 
is ,78, If the validity of this score is only slightly below its 
reliability, the portion of variance in A^^ due to random error is 
.5. Thus our method is equivalent to assuming that the normalized 
partial relationship between the true measure of initial endowments 
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We assume that the function, f^ (equation (6)), will con- 

■ ' j 

8 1st entirely of arguments relating to the social class and home 

background of the student, since school inputs could hardly 

: ' 22 ' 

effect scores on tests taken at the beginning of grade one. 



i 
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There is ample evidence that grade one achievement scores are 
associated with measures of student social class. See Bereiter, 
Gray and Klaus, Pasamanick and Knoblock. 






VI. MEASURING THE LEARNING ENVIRONMENT OP THE SCHOOL AND THE HOME 



f ; 
"t 



!| 



We aim to estimate the effect of school inputs on 



the value added of schools, in order to isolate the impact 



of schools, however, we must specify as fully as possible all 



of tho environmental influences on le,arning, that is, ideally, 
the homo and the students' peer groups, as well as the school. 

A complete specification of the model is particularly impor- 

* 

tant in view of the specification bias likely to arise 



because of the close association found in most samples between 
school and home environments which are conducive to learning. 



and vice versa. 



We may derive some suggestion of the relative 



lU Coots on learning of various dimensions of the individual's 



* • * 

environment from another study of identical twins reared apart. 



In this case we use differences in achievement scores (Stanford 



Achievement Tests) for paired identical twins as the measure 



of differential learning. The relationship between environ- 



ment and learning is suggested by Table 3. Even more than 



rhe analogous table for IQ differences, the educational en- 



vironment is of paramount_ importance ._ it alone e.s'plains more 



than HO |x^r cent of the variation in scholastic achievement, 
Uhil*' this is hardly surprising, the insignificance of tlic 
social and physical environment among genetically equivalent 
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Table 3 

The Effect of Environmental Differences on Scholastic 
Achievement Differences Among Paired Identical 

Twins Reared Apart^ 



Environmental Difference 

j 


Effect** 


t statistic 


’ i 

Educational 


.899 


7.69 


• ■ ' ' 

Social 


.024 


0.21 


Physical 


.001 


0.01 


. r2 ■' ■ . 


.82 




d.f. 
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a) Based on data of Freeman^ Newman, and Holzinger (1937). 

b) Normalized regression coefficient of the environmental 
difference measure in an equation predicting achievement 
differences among paired identical twins. 



• .. 

individuals is striking. ^ Of course, the environments in 

question may have been very poorly measured. Nonetheless, 
the finding alerts us once again to the dangers of speci- 
fication bias in equations with no measure of initial endow- 
ment, and suggests that much of the importance of social class 
in school learning apparent from cross-section studies may 

reflect genetic differences associated with the educational 

1 

and social characteristics of the student's .family. 

Let us begin by asking what aspects of the student’s 

• I ’ 

environment could have some effect on learning, h brief 
survey of the literature on learning suggests that the major 
.characteristics of an environment which will effect the devel- 
opment of school achievement (as well as general intelligence) 
include: 

a. the quantity of verbal interaction and communi- 
- cation with adults; 

b. the quality of verbal interaction and communica- 
tion with adults; 

C« the motivation for achievement and understanding 
in the environment; 

d. the richness of and degree of opportunity to 
explore the physical environment. 

23 

Alone they explain only .13 of the variance of achievement 
differences. 
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F 

% , Our medsures of these dimensions of the environment 

c fur from adequate, although data do exist which allow us 
to attempt an empirical implementation based on the above a 
pr ior i specifications. Moreover a number of reasonably well 
established relat^ions in sociological and psychological research 

I 

will assist us in implementing the model, 

( 

Beginning with the non-school environment, we may 

represent the quality of the verbal interaction between chilli 

and adult by a measure of the educational level of parents or 
,.24 

guardians. Family size, as well as the number of adults 

living at home, provides a measure of the quantity of inter- 
t action and communication, If we restrict ourselves to 

variables which can be regarded largely as exogenous, the 
motivation for achievement may be indicated by parental 
. attitudes concerning the importance of schooling, as well 

I , 

24 r 

On the importance of language models, see Olim, Hess, and 
Shipman, and Jackson, Hess, and Shipman, 

1 

25 

Anastasi, 

26 

Although we are not able to include this variable in our. anal- 
ysis below as we have no adequate measures in our sample, at 
least one study, which sampled the parents as well as the children, 
has confirmed the importance of parental attitudes toward schooling- - 
See Peaker, Of course, parental attitudes must depend in some 
degree on the particular school in which the child is enrolled. 

Thus parental attitudes are not unainbiguously exogenous. 
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an measures of the potential objective importance of education 
in the life of the student. The race of the student may, 
among other things, constitute a measure of these expected 
returns, for we nave compelling evidence .that the economic 
rctuLns to schooling at the eleBientary and secondary levels 
are significantly less for black than for white children. 



The nature of the physical environment of the home may be 
measured by the quantity of reading material in the home, the 
parents' occupation or income, or proxies for these variables, 
such as measures of the quantity of consumer durables in the 
home. Evidence of a relation between malnutrition (primarily 
protein deficiency) and learning difficulties suggests that 
measures of the physical environment may serve as proxy for 
inspects of the physical development of the child related to 
learning, particularly for very poor children. 

i 

/ A nun^er of authors have attempted to take account 

I 

of the home and social environment of the child by stratifying 
their analysis according to social class. Available evidence 



Weiss, Hanoch. Differences in family interest in schooling and 
its associated impact on children's motivation is in part a 
cultural phenomenon, likely to vary among ethnic groups. For 
convincing evidence in one case, see M. Gross. 

- I 
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For example, see Kies ling and the U.S. Commission on Civil 
Rights in their study of the effect of racial integratiion on 
scholastic achievement. 



.• ■ 

suggests that although this technique is certainly useful in 

reducing the inulticollinearity among the explanatory variables, 
it is a thoroughly inadequate representation of the non-school 
effects on learning. Peterson and DeBord^ for example, found 
that within two refined sub-strata (white and black lower 
class urban children in the southern region) variables mea- 
suring home environment and parent -chi Id interaction explained 
.56 (white) , and .66 (black) percent of the variance in achieve- 
ment scores. The predictive power of dimensions of home 

environment within narrowly defined social strata suggests 
that an analysis using no other control for social environment 

^ f\ 

will be subject to serious specification bias. 



We may proceed in roughly the same manner (although 
with less confidence) with the empirical implementation of the 
model of the school environment. The quality of the inter- 
action between adults and child may be represented by me a - 

I 

sures of the educational level or verbal proficiency of the 



29 

Of course, the Peterson and DeBord findings could result from 
collinearity between the home environment and school inputs to 
which the children were exposed. This is not likely to explain 
the entire result, however. Within a group of black sixth grade 
students in the third socioecnomic quartile in a large North- 
eastern metropolitan area. Levin found that, in addition to 
various school input measures, a number of home measures were 

significantly related to scholastic achievement. His findings 
.are as yet unpublished. 

3 0 

The strength of the measured relationship between school in- 
puts and achievement observed by Kies ling is probably due in 
part to this bias. 







teachers. The quality of the interaction may depend in some 
degree on school policies, which may be represented by a host 
of imperfect measures of such aspects of school environment as 
the breadth of curriculum, and the amount of extra-curricular 
“ activities. Tile physical environment of the school may be 

irepresented by .measures of special facilities (labs, libraries, 

• etc. ) . 

i , , . 

Table 4 summarizes our model of environmental influences 
bn learning, and our proposed empirical implementation of the 
model. 

Notice that even this partial specification of the learning 

\ ‘ ' ' I • ’ . ‘ 

environment includes 14 measures, many of which are highly 
correlated. Thus serious multicollinearity problems arise in 

the estimation of a full model of the type specified, in order 

to estimate the | above model, we need to reduce the number of 

' - 0 ’' 

variables so as to simplify the presentation and bring the 
multicollinearity problem within tolerable limits. That is, 
we would like to replace the equation 
(9) 

A — f(Xj^, a.., ) 





j ! 



by 



( 10 ) 



A ~ F •••# 0 •••# ^v ) » •••# 



... -v)J 



where h < v. 
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Thus we may wish to define a new variable, say "teacher 
quality" as an aggregate of individual variables measuring 
the teacher's verbal ability, years of schooling experience, 
certification, and so on. If a significant degree of multi- 
collinearity arose from intercorrelations within the set of 
variables which form the aggregate variables, the problem 
will be reduced and the new synthetic variables, represented by 
gi • • • 9^ sufficiently orthogonal to allow successful 

estimation of the relationship. The precise grouping of 
factors is, of course, determined by more than the desire 

i 

to reduce multicollinearity, although the usual aggregation 
rules do not seem particularly helpful here, as we have ab- 
solutely no knowledge of the matrix of second derivatives 
and cross-derivatives which would allow us to make use of 
them. 

^ ’ 4 * * 

0 

We have no previous results or compelling theory 
which provide guidance in how to aggregate. In situations in 
which all inputs are priced in the market, and the assumption 
of maximizing behavior is somewhat more plausible, we ordin- 
arily use factor or commodity prices as the basis of aggre- 
gation,' as in the measurement of "capital” or intermediate 
inputs. Failure to appreciate the importance of these assump- 









^ 

tibns in the validity of any monetary aggregate in production 



theory has lead to the frequent use of what might be called 



spurious factors in the analysis of school inputs, such as 
expenditure per pupil and teachers' salaries. In my own 
estimates, (for black twelfth graders) instructional expend! 



ture per pupil 



is in virtually no case significantly related 



to achievement in a properly specified mode. Yet most of the 

r 

factors which are purchased with the expenditure, and which 
account for its variation, such as teacher quality and school 



Facilities, show a strong relationship with achievement. Sim- 
ilarly, whereas teachers' salaries alone explain only .0085 of 



the variance of achievement, the two factors most closely re- 



lated to varaiations in teachers' salaries -- teachers' verbal 



abilities and years of schooling — explain over four times 



as much.^^ All. of. this simply suggests that school administrators 



are using their resources efficiently as far as the production of 

' oo 

scholastic achievement is concerned. Thus the use of monetary 

i 

aggregates is unfounded in theory. 
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In each case I am referring to the increase in the coefficient 
of determination in an equation already including measures of 
social background and non -teacher school inputs, as in equation 
(5) on page 22. See Levin for an analysis of the relation between 
teacher quality and teacher salary. These two teacher attributes 
(verbal ability and years of schooling) explain 60 percent of the 
variance in teachers' salaries. 
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This inference is supported by a comparison of the estimated 
marginal products (f. ) and the supply prices for various teacher 
attributes. (See Levin). Calculations of the cost of unit in- 
crease in achievement through increases in each factor based on 
these estimates show that for the sample under consideration, 
increases in teacher's verbal ability are more efficient than any 
other dimension of teacher quality, by a wide margin. 
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In our situation the best available method seems 
to be to attempt to identify the underlying dimensions of the 
input structure, both fay a priori and empirical methods. 
Having done this, we would like to select a variable, or an 
index based on a number of variables, to represent each dimen- 
sion. Our a pr ior i specification suggests that we have 
roughly four important dimensions: teacher quality, teacher 

quantity, school .policy, and physical facilities. One pro— 
cedure would be to assume that these represent the dimensions 
input structure, and to select from each set a variable 
to represent the underlying input. Thus it would be plausible 
to represent teacher quality by the teacher's score on a 
verbal ability test, at least when we are predicting verbal 
achievement, and so on. 

However, we may combine our preconceptions based 
on previous research and learning* theory with an empirical 

/ f 

analysis of the structure of our data, using principal com- 
ponents analysis. Although our results below are gener- 



ic 




ated without the aid of principal components analysis, i am 
currently experimenting with this approach. 



i Leaving the problem of aggregation in this unsatis- 

i ' 

factory state, let me ask how well we have measured the envi- 
ronmental influences on learning, particularly as they relate 



This is the method used by Kies ling. 
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to the school. The answer "not very well" stems primarily 
from three problems; a) our home and school variables fail 
to capture the complexity and richness of the interactions 
processes which are relevant to learningf; b) we have ig~ 
nored significant differences in the education offered within 
the same school; and c) we have measured inputs at only one 

* < t 

point of time, while the learning process must certainly be 
cumulative and therefore depend in some degree on past as 

well as contemporary inputs. 

Turning to the first objection, our measures of 
social class, family size, class size, teacher quality and 
school facilities do not measure the quantity and quality 
of interaction as relevant to learning, but provide only 
crude measures of a few of the opportunities for it. Two 
recent studies suggest that the crude measures are a poor 

' ■ t 

substitute for measures of actual observed patterns of inter- 
action. On the basis of highly detailed interviews with 60 

parents, both Dave and Wolf found that their measures of home 
environmental effects on intelligence and achievement explained 
.57 and .64 of the variance in the attribute measured. The 

I 

^"^Recall also the Peterson and DeBord study, op. cit .i 
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it ^rude home environment measures used in our study explain about 
I 10 percent of the variance in individual achievement scores. 

presumably analogous studies of actual classroom interaction would 

I 

reveal that our school measures are a poor representation of our 
basic learning model. 

I The second set of problems arises particularly where 

tracking is widespread and the differences in the education 

received within the same institution are so great that we 

. . 35 

really have two or three schools within the same building. 

I Moreover, differences in teacher and administrator attitudes 
and expectations toward children differ considerably within 
a school and even within a given classroom. One recent 

study (Rosenthal and Jacobson ) suggests that teacher 

expectations have a significant effect on learning, at least 
in the early years of school. The specification error intro- 
duced by the failure to measure these within— school and within- 

i . 

classroom differences in inputs is particularly serious be- 
cause of the correlation of these differences with other of 
our explanatory variables. Because low social class and 



minority racial or ethnic status are closely associated with 



Differences in the quality and quantity of school inputs received 
within the same school are documented in Hollingshead. 

I 
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See Davis and Dollard, pp. 284-285, and Warner, Havinghurst 
and Loeb, as well as more recent studies by Deutsch and Wilson. 
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j.a within-school deprevation of school inputs, 37 the estimates 
of the parameters reflecting the impact of social class and race 
are biased upward* Further, because of the serious errors intro~ 

p 

f 

duced by the school-wide aggregation of the variables measuring 
school inputs, the estimated effect of the school environment is 

OQ 

biased downward* 

Our third objection, against the sole use of conteinporary 
inputs measures, would not be serious if children did not move 
from school to school, and inputs were roughly uniform throughout 
all of the grades up to the one for which the production function 

I ' 

is being estimated* Of course, the world is simply not like that, 

t 

and 1 think we sometimes underestimate the seriousness of this 
problem* In a sample of black sixth grade students in a North- 
eastern metropolis, 57 per cent had attended more than one 
school since grade one, and 29 per cent had attended more than 
two* Evidence from a number of studies of the phasing of 
learning development over the school years suggests that this 
problem is particularly serious, as patterns of achievement 
SiTCQ apparently established with a high degree of stability in 
the early grades* Scannell $ for example, found that 



^^See the evidence in Hollingshead and the more recent studies 
cited in Rosenthal and Jacobson* 
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In a study in which within-school variations were measured, 
Peaker found that school inputs were considerably more impor- 
tant in the determination of school achievement (relative to 
other influences, such as home background) when within-school 
variations in these inputs were taken into account* 

39 

Work in progress by Henry Levin and Stephan Michaelson* 
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.scores on fourth grade tests (Iowa Tests of Basic Skills) 
explained half the variance in test scores (Iowa Tests of 
Educational Development) in the twelfth grade Cardinal 
measures of scholastic achievement based on vocabulary tests 
suggest that about two-thirds of what is known in grade twelve 

was already known in grade six. On the presumption (which 

0 

seems to have currency among educational psychologists) that 

the effects of environment on learning are potentially greater 

. ’ > 

during periods in which the most learning takes place/ it 
would seem that measurement of the inputs in the early grades 
would be essential to the prediction of achievement at the 
higher levels 



The relative importance of the early years in the 
learning process suggests one last question: how much impact 

on measured learning can we expect schools to have? During 
the elementary and high school years, children ordinarily 
spend considerably under a quarter of their wakeful hours in 
school. Moreover, Bloom (1964 suggests that about a third of 
adult learning is achieved before age six. His survey of the 

t ' 

impact of extreme environments on learning suggests that we 
might expect changes of 1.25 standard deviations on the usual 
tests due to environment from ages 0 to 1« which is consistently 
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Scannell; Bloom summarizes the evidence on the stability of 
achievement. 

41 

In the absence of a time series of school inputs, it might be 
advisable to concentrate on the estimation of production relations 
in the early grades. 
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very conducive or prohibitive to learning. And if the school 
environment is applicable to an age span in which only two- 
thrids of the learning takes place, and at that for only part 
of the time, we might regard an impact of less than a standard 
deviation as an expected effect of a very good or a very bad 
school as opposed to an average one. 





I 



I 









ipimjiPiiapivpffiwpiPii 






— ' 1 ' 



' VII. THE PROBLEM 0? NON-RESPONSE 



TO complete the survey of problems in the estimation of 
educational production functions, i turn finally to the statisti- 
cal shortcomings of the available bodies of data. Although I 
will later make use of data from the Office of Education's Equality of 
Educational Opportunity Survey, i will not undertake specific 
analysis of the statistical properties of these data,^^ Rather, 

I concentrate on the data from the main sample used here — — 

the males who were high school seniors in 1960 and who responded 
to both the initial 1960 project Talent survey and the five-year 
follow-up survey. i will consider three distinct sets of prob- 
lems: non-response of schools in the initial survey? non-response 

of individual students in the five-year follow-up? and non-response 
on particular survey items by individuals returning the follow-up 

questionnaire. 

¥ 

a. Non-response by schools. 

The sample is based on a list of public senior high schools 
(i.e,, all schools including a 12th grade) compiled by the Office 
of Education. These schools were sorted by states and arranged 
the nine U.S. Office of Education Regions. The five la rgest 
cities — — New York, Chicago, Philadelphia, Detroit, and Los 
Angeles were treated as a separate Region. Within eachTstate, 
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The interested reader is referred to Bowles and Levin and Mayeske. 
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schools were sorted into four size categories : 



Number of Students in Grade 12 



1 0 - 24 

2 25 - 99 

3 ■ ■ 100 - 399 

4 400 and over 



Differential sampling ratios were employed in order to get 



-sufficiently large sample of large schools. A random sample 
of 1 in 50^ 1 in 20 , 1 in 20 , and 1 in 13 was drawn from each of 
the four size categories, respectively.^^ Of the 1,063 senior high 



response is a relatively minor problem in the Project Talent data. 
b« Non'^response by individuals in the five-year follow-up. 

There were 30,165 male seniors sampled in 1960. Only 15,975 
responded to the five-year follow-up survey. The 47 percent non- 
response rate alone is enough to cast serious doubt about the 
usefulness of the data. Moreover, we have reasonably good evidence 
that the pattern of non-response is not random. Table 5 presents 
data on the distribution of the talent five-year follow-up respon- 
dence by race, region, occupation of parents, and urban/rural 



The above description is based on Flanagan (1962). 




schools selected in the sample, 987 eventually returned usable 
data. This 93 percent response rate suggests that school non- 



■ ; * « ‘ 

M V ■ ■ 

; ( * • . 

; i Vf residence, fhese data are contrasted with the distribution of 
enrolled 16- and 17-year-olds in 1960, according to the U,S. 

!; Census. The correspondence between the Talent definitions and 

! 

the Census definitions is not exact. Moreover, we are unable 
to determine the extent to which the discrepancies between the 
Talent distribution and the Census distributions are the 
result of biases in the initial Talent sample, as opposed to 
non-random non-response on the five-year follow-up. Nonethe- 
less, the data in Table 5 do indicate that the males on the 
Talent five-year follow-up sample are not representative of the 

t 

total population. The discrepancy for race is particularly 
serious, there being only half as many blacks in the sample as 
would have been expected on the basis of a random sample. Of 
course, the biases indicated in Table 5 may be surmountable by 
the careful use of a weighting scheme, or preferably by strati- 
fication of the sample. 

I 

But it would be fortuitous if the non -randomness of response 

j . ‘ 

I were limited to these variables for which weighting is possible 
and stratification a plausible procedure. There i strohg"evi- 
dbnce, for example, that the black respondents to the five-year 
follow-up are characterized by higher levels of scholastic 

I* 

achievement than would be expected on the basis of a random 

I 
r 

sample. Data from the Equality of Educational Opportunity 
Survey (EEOS) allows us to calculate the gap between blacks' 

i 

i. > 

r 
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Table 5 

Comparison of the Unweighted Project Tilent Sample (5-Year Pollow-Up 
of Male Seniors in 1960) with National Population 



Characteristic 


% in Sample 


% in Population^ 


White 


.968 


.921 


North 


(USOE Regions)^ 
. 1 
Rich^ . 

(classified by 


.760 


.730 


parents' occupation) 


.603 


.581 


*^Based on school enrollment 


j 

1 

of 16— and 


17-year-olds. U. S. Census 



1-960. PC(2)5A 

Occupations in Census : 

Rich: skilled worker, foreman; clerical worker; salesman; 

manager, official; proprietor or owner; professional, 
technical 



Poor: farm owner and/or manager; farm foreman; farm worker; 

workman, laborer; service worker, including household; 
protective worker; semi-skilled worker 

Occupations in Talent 



Rich: skilled worker, foreman; clerical worker; salesman; 1 

official; manager; proprietor or owner; technical 

'poor: workman, laborer; farm, ranch foreman; farm, ranch 

worker; private household worker; protective worker; 
service worker; semi-skilled; don't know ^ 

^Regions classified as "North": Census: northeast. North Central, = 

and West 

Talent: USOE regions 1, 2, 4, 7, 

8 & 9 - New England, Mid-r 
east. Great Lakes, Plains, 
Rocky Mountains, West, and 
*^on-continuous states 1 
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test scores and the average scores. Similar comparisons may be 
made with the Talent data, using the reported scores from the 1960 
survey and the scores recorded for blacks on the five-year follow- 
up. These comparisons (using. Northern Blacks) are presented in 
Table 6. The unmistakable inference is that the Talent follow- 
up sample blacks are achieving much closer to the national mean 
than are blacks generally. 
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Table 6 

Relative Black and Total Scores, 

Equality of Educational Opportunity Survey and project Talent Five- 
Year Follow-Up of 1960 Male Twelfth Grade Students 



Tests 



Difference 
Between Average 
Score and Northern 
Black Score^ 



Standard 
Deviation 
of Northern 
Black Score 



Difference^ 
in Black 
Northern 
Standard Devi 
at ion Units 





(1) 


(2) 


(3) 1 


Equality of Educational Opportunity Survey^ 




(l)/(2) ' 


Verbal scale Score 


16.3 


14.4 


• # 

1.12 


Non-verbal Scale Score 


9.1 


8.2 


1.10 


Project Talent 








Reading Comprehension (R250) 


4.8 


11.5 


.41 


English Total (R230) 


5.4 


l3.9 


.38 


Abstract Reasoning (R290) 


1.2 


3.05 


.39 


Math I (R311) 


2.3 


3.4 


.67 


General Academic Ability (C002) 


66.1 


120.1 


.55 





li 










Notes to Taoie o 









2^ .WM. 




^Equality of Educational Opportunity Survey test scores are from 
the Appendix of the Report . 



y% 

The average score on the EEOS tests is a weighted average of the 

total white score and the total black score, using as weights the 

i 

fraction of white and non-white 16- to 17-year-olds enrolled in 
■school in 1960. The omission of non-black non-whites results in 
a very slight underestimate of the average score. The average score 
for the Talent tests is from Flanagan (1964) table 13-2, and 
refers to all students taking the Talent test battery in 1960. 

The "Northern" Talent scores refer only to USOE regions 1, 2, 

I . . . 

} 

and 3. To some small extent, the discrepancy in scores is due to 

I 

I 

our representation of "Northern" by regions which are roughly 
"Northeastern." Evidence from the EEOS suggests that black 
students in the midwest and west in metropolitan areas score about 

. 1 of a standard deviation below blacks in the Northeastern 

0 ' • 

metropolitan areas. 
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c. Non-response on particular survey items. 



Not all of the students returning the five-year follow-up 
questionnaire furnished all of the requested information. 
Moreover^ there are a substantial . number of missing responses 



student information questionnaire administered in 1960. 
Table 7 , which summarizes the extent of the problem, contains 
information on the number of respondents with no missing data, 
and those with various numbers of items unanswered. The 

amount of non-response by questionnaire items is also recorded. 
The degree of non-response is substantial, and again, we 

have compelling evidence that the pattern is not random, un- 
fortunately, there is no follow-up of the non-respondents from 
five-year follow-up. However, for this particular group we 
can infer a non-random pattern of non-response. A comparison 
of achievement test scores indicates that those not responding 
to questions concerning their parents* occupation, education and 
other dimensions of their social class scored on the average 
lower than those who did respond. 
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Nunnibcr of R6spondGn'ts by fbc NunibGir of Missing ObsGrvfltions 

\ 

1960 Male Seniors in Project Talent Five-Year Follow-up 



Nundber of variables with 
Data Missing 

(Total Number of variables 
« 124) 



( 1 ) 



1 


or. 


less 


5 


or 


less 


10 


or 


less 


15 


or 


less 


20 


or 


less 


25 


or 


less 


30 


or 


less 


35 


or 


less 



Cumulative Number of 
Respondents 
(Total Number = 15975) 

JT) 

345 

2049 

3373 

6849 

10485 

13571 

14780 

15124 



i 
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Table 8 

th Missing Data on Selected variables 
Project Talent Five-Year Follow-Up 

Total Respondents = 15,975 

Variable Number Number of Cases with Missing 

■ ^ r‘" ' Data 



Number of Respondents w] 
1960 Male Seniors in 



7 Class size, science and math 128 

10 Senior class size 162 

17 Educational innovation 319 

21 Starting salary, male BA with no 302 

experience 

25 Percentage of teachers fully 198 

certified 

27 Percentage in college preparatory 841 

45 Regular part-time teachers 325 

52 Tracking ‘ 421 

54 Percent of Blacks 1201 

67A Father's occupation 1291 

67b Mother's occupation 1159 

68 Father's education 1274 

69 Mother's education 1182 

71 Own room, desk, typewriter 997 

72 Appliances 965 

73 TV, telephone, radio, phonograph 1015 

85 With whom living 1078 

103 Starting salary - monthly 3952 

104 pay on October 1 - monthly 4006 



107 



Race 



336 



*. 

various approaches to the missing data problem have been 
44 

proposed. Where the number of respondents with missing data 

Is small, all respondents with missing, data may be eliminated from 

• ) 

the analysis. This method is clearly inappropriate here, as it 
. would drastically reduce the number of observations, to some 
extent unnecessarily, as the number of variables retained in the 
final analysis can be expected to fall considerably short of the 
number with which the analysis is begun. Alternatively, one may 
estimate the regression coefficients of y = xb from the rela- 
tionship: 

(11) cov(x^,Xj)b = cov(x^,y) 

where cov(x.,x. ) is the covariance matrix in which the (ij)^^ 

J 

element is calculated on the basis of observations for which data 
on both i and j are available, and similarly for cov(x^,y), and 

A. 

b is the vector of estimators. This method is more flexible in 
that it allows experimentation with all variables for all observa- 
tions possible. It is particularly appropriate in attempting to 
arrive at a correct specification of arj equation when there are a 
very large number of candidate variables. However, Haitovsky's 
Monte Carlo studies have shown that this method yields seriously 
biased results when the number of non-responses is high and the 
pattern particularly non-random. Thus, although this method was 
adopted in this study, the results must be regarded as provisional. 
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See Haitovsky 
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♦ 

As these estimates yield considerable insight on the correct 
specification of the educational production function, further 
studies, operating with far fewer variables, probably should 

adopt one of the many methods of assigning values to the missing 
data. 

In Table 9, I present data on the number of respondents for 
which complete information was collected on each pair of the 
main variables used in the analysis of the Project Talent data 

' - ' _ ' II ' . • • • 

for Blacks in u.s. Office of. Education regions 1, 2, and 3. 

No analogous data is avail^le for the data from the EEOS. 

,/ II . , - ■ ^ ^ ■ - • -• - . 
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irxxx. PRQV'XSIC^flL RESULTS^ THB IMPORTANCE OF SOCIAL CLASS AND 
THE scaboL 

■ M 

t' ' 

X have chosise thire<^ different measares of output : reading 

coaprehensionff irteroiediate high school mathematics competence, 
and a composite score based on a number of tests. The reading 
OGsprehension test measures idiat is commonly known as 'academic 
intelligence, * and is a good predictor of school success in an 

ij 

..II ^ ^ 

ac^emic or„ liberal arts curriculum. The mathematics score is 

the sum of two test results. Math IX and Math III. The first 
aeasures achievement in the mathematics generally offered up to 
and including the ninth grade. Hath lit covers topics normally 
included in tenth to twelfth grade mathematics courses, particu- 
larly in college pireparatory curricula. Whereas the material in 
the reading comprehension test could easily be acquired outside 
the school, it seems reasonable to assume that the abilities 
measured in the two mathematics tests are learned in the class- 
room. The composite test score . — General Academic Aptitude — 
is baised on nine individual tests, as listed in Table X0« 

All of the variables aj^iearing in the following equations 
auce listed in Tables 11 along with their means and standard devi- 
ations. A table of zero order correlations appears in the 
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Mesins and Standard Deviations of varipd>les 
Biack Male Twelfth Grade, USOE Regions 1,2,3 
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variable Name 


Mean 


Standard Deviation 


Dependent Variables 






. Reading Comprehension 


28.07 


11.54 


General Academic Ability 


473.39 


120.14 


Math II and III 


.84 


.72 


starting Monthly salary 


369.12 


189.28 


Home Variables Indieles 




‘ ..****»■ ' • . ' ■ * * 


• Father *s Occupation 


6.07 




Mother's Occupation 


4.77 


4.52 


Father's Education 


3.52 


2.58 


Mother ^is Education 


3.64 


2.20 


Own Room, Desk, Typewriter 


11.51- 


1.00 

j> 


Appliances 


114.76 


12.45 


TV, Telephone, Radio, Phono- 
graph 


13.22 


1.11 . 


With Whom Living 


.27 


.44 


School variables 






Class Size, Science and Math 


23.77 


3.94 


Senior Class Size 


447.48 


286.02 


r 

Educational innovation index 


8.94 


1.04 


Starting Salary, Male, B.A. 


4448.07 


438.92 


9^ Teachers Fully Certified 


96.47 


11.03 


% in College Prep (plus 100) 


136.17 


23.00 


Teachers Grad Tr a ining/c lass 


.49 


.59 


Tracking 


.86 


.34 


% Negro 


45.87 


38.54 



• 7 - 



a 




The estimate of the educationcil production function for 
each of these output measures appears in Tables 12-14. The 

following aspects of the results are important. 

The importance of social classy in each equation measures 
of the student “s family background are highly significant, two 
variables appear in all three eq^iations — the occupation of the 
father, and a measure of consumex: durables in the home. The 
occupation of the father is the value of ah occupation index 
scaled according to the mean income in particular occupations. 
Table 15 describes the scaling method. The consumer durables 
variable is the sum of yes responses to questions concerning the 
presence of a television, radio, telephone, and phonograph in the 

home. Both the father ' s occupation and the consumer durable 

I , . • 

v’ariables are measures of family income, it is interesting, to 

note that these income proxies explain scholastic achievement 
better than variables relating to the parents * education. Only 
in one of the three equations is a measure of the parents' educa- 
tion — - the mother's — significantly related to achievement. 

As expected, the f&mily background variables are more 
closely associated with achievement in reading comprehension than 
in intermediate mathematics, in Table 16 the sum of the beta 
coefficients relating to background as opposed to school charac- 
teristics are reported. 
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Table 13 

An Educational Production Function 
Dependent Variable is Mathematical ^achievement 

Black Twelfth Grade Students 






Independent variable 



Regression Coefficient 
(t in parentheses') 



Beta 



I.,:; 


path€ir ®^s Occupation 


0.0267 

(2.4970). 


0.1562 




'Educational innovation 


0.1164 
• .(2.6506) 


6.1684 


' w 

'M* . - •, ♦ f .. 

‘ : 


f^Tracking \ ;;,'y 


-0.2533 

■•(-1.9089) 


-0.1213 




TV/ Telephone, Radio,, Phono- 

graph 


' .1986 

(4.8456) 


0.3065 




Expenditure per Student on Non - 
, Teaching inputs^' 


.0007 

(2.6617) 


0.1711 




Teachers with Graduate Training/ 

: class , 


.2227 

(2.8029) 


0.1820 




Age of Building 


-0.0069 

(-2.4309) 


-0.1559 


• '/ V> , ■ ■ * ^ ‘ ^ 

donstant -2.1B981 

(-4.1429) 






R^c 


L,.'-,,-. 0.2199 : ■ 




- ; 


|x’ 


x\ .7747 







number of observations see Table 9 



Expenditure per student on non— teaching inputs is a measure of 
expendxture per student mxnus a measure of the per student starting 
salary of a male fully certified teacher. 
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Table 14 

An Educational production Function 
Dependent variable is General Academic Ability 



independent variable 


Regression Coefficient 


Beta 


} 


(t in parentheses) 


* . • 



constant 



R c 



-43.7764 

(-^0.5501) 



0.3275 

\x'x\ : 

number of observaticins ' see Table 9 



1. 


Father's Occupation 


5.9546 
(3.4707) ■ 


0.2082 


_ '! • 

7. 


Educational innovation 


8.5838 

(1.2645) 


6.0742 


5. 


Tracking ' , 


-47.1432 

(-2,0648) 


-0.1350 


2. 


TVi Telephone, Radio, phono- 


; 41.9373 

: (6.5982) 


0.3870 


3. 


Mother * s Occupation 


2.5075 

(1.5752) 


0.0943 


6* 


Class Sise, Science and Math 


-3.9846 

(-2.0278) 


-0.1307 


4. 


Teachers ik/ith Graduate Training/ 

Class - 


26.7^28 

(2.1930) 


0.1307 



' .• ii 
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The imp.^rtance of the scaiool, 

a. Teacher Quality. ^ variable aeasaringf teacher quality 

the number of teachers with graduate training divided hy the 
number of classes in the school — - Is significantly related to 
scholastxc achxevementf aithou^^^ xn the produMyt io n of reading 
comprehension the relationship is rather weak. *r*h-Big variable 
is such a poor measure of idiat nahes a good teadher that our 
results give us only the vaguest idea of the iagiortance of teacher, 
quality as a determinant of s^cdiolastic aciiieveimfntt. Our regresslom 
coefficients are surely underestimates. 

b. Class Size. The aveitage slse ot classes in the science 

and mathematics subjects is related to scholasl:;ic achievement fOr 

« *: 

cf our three cases • The variable is« of ccnirse^ a general 
measure of class size, being cbr^lated fr^* .45221 with 

Class size in other subjects. Nonetheless, it seems someiiliat 
ahomolous that the class size variaible is not significantly 
eclated to mathematics achievement. 

c. Educational innovation. An of educ»tioi»l innova*- 

tion was constructed to measure the extent of innovation in both 
curriculum and equipment, ahe index is based on the responses 
to three items on the General scdiool Characfceristics Questionnaize 
which was completed by the school principal or his staff: 



1 * "In wl'iich of fHe following areas has your grades 9^12 
sicliool. falcen pairf xn a large scale lnt:er~syst:eiii liryout: of a 
spepxal ex)^riment:al curriculum?*' (response by subject, area,,' or 
*^none**) 

2. "m DliLicb of the following areas has your graders 9-*12 
school or school system developed and tried out its own special 
cssqierimental ctirriculum?** (response by subject area, or "none "I 
3« "Ittiich of the following statements best describees tlhe 
cnirrent use (grades 9—12) of teaching machines in your selhooll? 

'1, i ; 

(Teaching machines may be thought of as individual self-instruc- 
tional devices which automatically provide both learning mateirial 
and answers to student responses • They do not include the usual 
eduOational films, slides, educational TV, etc.}** (response by 
level of use, including "none") . 

The "none" responses to the above three questions were sunuMid 
and subtracted from ten to yield our index of educational innova- 

. ? I - ’ 

s “ ■ 

* ■ ’“ill 

tion. In one of our three cases, educational innovation is sig- 

1) 

®ificantly related to achievement, it will be seen below that in 
a second case (the production of general academic ability), 
educational innovation variable is significant in a aiore &lly - 
specified equation, it is likely that some of the apparent 

ill 

influence of educational innovation is a reflection of general 

III 

school atmosphere, the innovative schools being more open a nd 
experimental generally, not simply with respect to curriculum and 
equipment. 
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4« EcoRonies of Scale. In no case did the addition of a 
vaciahle neasniring ildie senior class size yield a statistically 
aigniflcant increase in the fraction of variance explained by 
the equation. Table 17 records the fc-statistic for the class 
size variable ^en added to eacdi equation^ along with the size 
of the estimated (Kxsfficient. Similar results were generated 
using the EBOS data. The fact that the estimated effect of 
cdianges in senior class size is insignificantly different from 
zero is in conflict with much of the literature on econpmies of 
scale in secondary {schooling. It should be noted that the 
senior class size is a reasonably good proxy for size of community, 
iSsns our result may reflect a combination of genuine economies 
of scale which aire offset by unmeasured negative effects of the 
center city school environment. A non—lxnear relationship between 
senior class size and achievement %iould seem plausible even if 
tile cosmunity size loould be accurately controlled in the equation. 

■ 

I have not estimated the relationship with a non-linear senior 
cla^ size variable. 

5» School policy: tracking. The measure of school 

tracdcing is a dummy variable indicating that tracking exists if 
tiie school has two or more tracks. (A single track with electives 
is not regarded as tracking.) in all three cases, tracking is 
negatively associated with scholastic achievement. The predicted 
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See the work of J. Riew, £. Cchnc and H. J. Riesling. 
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T^le 17 



t~Statistics for Senior class size Variable 



Estimated Effect 

dependent Variaible t-statistic of increasing class 

size on Achievement 



Mathematical Achievement 0.6927 

" 4: r. ^ 4 ^ 

^Reading C!omprehension . 0..1464 

General Academic Ability 1.4199 

1-. :... . ~ ' - i- - ■ ^ 





level of scholastic achievement in schools with tracking is .3174 

I ' * 

,3518, and ,4 08 5. standard deviations below those which do not on 
the reading, mathematics, and composite score, respectively. Two 
interpretations of this result come to mind. Both begin from the 
presumption that black students are likely to be on the average 
lower achievers, and of lower social class than their white 
school mates • The firs t interpretation is that tracking has a 
negative effect on scholastic achievement in that it minimizes 
the contact between the students in our sample and the higher 
achieving^ higher class students ini the rest of the school. 

This interpretation can be tent at ifvely rejected, as we would in 
this case expect ^that the achievement and/or class composition of 
the school would .produce a positive effect on achievement, holding 
constant the degree of tracking. That this is not t|he case is 

• ' . ‘ ^ -'j- . i ' 

indicated by the fact that, when sidded to the above equations, the 
variable measuring the percentage of children in the sphool in 
the college preparatory subjects is never significantly related 
to achieve^nt. , 

A more compelling explanation is simply that th€) level of 
resources devoted to a child's education is not uniform within a 
school — it varies in part according to the track the student is 
in. This l^ing the case, and recalling that blacks are dispropor- 
tionately likely to be placed in the "slow" or otherwise dis- 
advantaged tracks, we may interpret the negative coefficient of 
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tracking as reflecting the influence of unequal school resources 
within schools • in schools with tracking, the black students , 
on the average, receive a level of dchool resources which falls 
short of the school average. This is reflected in their 
-achievement : scores* • ■ 



6.« , School Policies.: Integration. When we add a variable 




^rcentags 



student body which is black, we 



results in Tables 18 through 20. in two of the three 
cases thore ^ significant negative relationship between the 
level of achievement by sample of blacHc students and the pro- 
]^rtion of the- student udiich is black. Given the fact that a 

measure of the sdcial class and achievcbment levels of the school 
- (percents^ ihj college pr€»paratory sub is not significantly 
I related to black achievem^fnt, it is difficult to interpret this 
result as a peejr effect iii^olving the transfer of "good " learning 
habits, ^^1^ etCi, from the high achieving Whites to 

the Ic^ achieving blacks* I alternative (untestable ) interpreta- 
tion is that the apparen^^ the proportion of blacks in the 

school arises from the fact that the social backgrounds of black 
children in integrated schools and those in all black, or nearly all 
black, schools differ in ways which are relevant to learning but 
which are not captured in our crude social class measures* The 
results cannot be interpreted as suggesting that school integra- 
tion will raise black achievement . 
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Table 18 

Aii Educational production Function 
Dependent variable, Reading comprehension 
with Integration Variable 
Black Twelfth Grade Students 
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1 ij L - II mrnm .m u- 


Independent Variable Regression Coefficient 


Beta 


■' 'v ' 


; (t 


in parentheses ) 




!• Father's Occupation 


0.5503 


©•2003 






(3.1162) 


’ • ’ . , c'-‘ 


TV, Telephone, Radio, Phono- 


2.9622 


0^2846 




graj^ 


(4.5124) 




, * i , t ^ ' r ■ 

$• Mother's Education 


0.6634 


0.1262 






(1.9363) 




4 • Teacher ' s Graduate Training/ 


1.9449 


0.0989 


Class 




(1.5924) 




5 • Tracking 




-3.7356 


-0^1il4 






(-1.6221) 




6^ Class Size, Science & Math 


-0.4111 


-0^1405 




1 


(-2.0959) 




7^ Percentage Black 


-0.0253 


-0^0846 






(-1.3636) ■ 




Constant 


-1^5930 







R^c 



jx*x I 

number of observations 



(-0,1611) 

• 2476 

• 6127 

see Table 9 



ERIC 



'k • ■ 



i 
























An Educational Production PunctiCn 

Var icible is Mathemat i.c Aclliieveraent 
with icnteifration variable 
Black We If Ith Grade Studentli 



number of 61>servations sei Table 9 



ERIC 



' ’’ ■ . i ■ ' 




Independent variable 


Degression Coefficient Beta 

; ;,(t in.iparentheses) , 





Father *s 'bccupation 


0.02)7 


0.1384 








(2.2301) 






TV#. Telephone, RadiOj Phono- 


^■::;%0io.l9]L2 . 








;|:.graph;;^ 


::v j:(.4. 71133) 






AgO' of Buildi 




: ' 5 "> i0.P044;^'''^ 
'‘-/'C:;(-a.52|8| 


-0.1000 


' 47 " 


Teacher *SGra 


iduate Tijaining/ 


^6^115:'' ■ 


0.1728 








(2.6952) 






Tracking 




-0.2643 










(-2.015) 






class Size, Science & ! Math 


0.0006 

;v"vf;(2i45i3): 






Educational innovation 


0.1504 


-0.0404 








, (3.3212) 






. Percentage Black 


•^^■-0:.003i^:■■^■ 


-0., 1751 








(-2.5726) 




Ji^M:'Cbnstant,’ 


-2.9744 : 










(-4.3076) 










0.2413 


■ , , , , '■ . ■■ ■’ ;; - , 




X*: 


t\ 


(1.6126 













Table 20 

An Educational production Function 
Dependent variable is General Academic Ability 
with Integration Variable 
Black Twelfth Grade Students 



Independent Variable Regression Coefficient Beta 

(t in parentheses) 





Father's Occupation 


5.3193 

(3.1216) 


0.1800 


' - ■ r 

2. 


TV, Telephone, Radio, 
Phonograph • 


'41.0042 

(6.5539) 


0.3788 


'3.'- 


Mother's Education 




2.9140 

(1.8522) 


0.1096 ! 


4. 


Teachers with Graduate 
Training/class 


25.9364 

(2.1600) 


0.1267 i 

1 


5. . 


Tracking 




-51.0832 

(-2.26.91) 


■"1 

-0.1463 


6. 

f- 


Class Size, Science & Math 


-3.8132 

(-1.9711) 


-0.1251 J 




Educational innovation 


14.2960 

(2.0433) 


0.1236 


i . ' " ‘ ' 

! ' ; 8. 


Percentage Black 


■■■■■ ' • ' 


-0.5107 

(-2.7441) 

■j 


-0.1638 

■ A 

... 


"Constant v 

lx*xl ^ 

nunibeir of <^servations » 


-78.8240 

1-0.6901) 

0.3488 

.5536 

see Table 9 
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IX. 



C^J^RISW OP RROJECT TftLElsn? AND EQDALrEY OP EDUCfijJriQlSlAL 
OpkiRTiniiTy PURVEY RESULTS ' 



(■ , r , 

order to assess tllie generality of these results, I will 



hpiw estiniate similar functions with a different body of data, i 
The following estiiptes are for black students who werel enroll 



led 



in twelfth grade in the fall of 1965. The data were coll4cted 



by th€ 

, ... . 

Qipport 



U.S. Office of Education as part of the Equal Educational 

■ ' '■ '* " ■■ . ■ ' ■ • . ■ :■ ■■■■ f ' ./■ 

unity Survey. , Some! of the results of this survey ji havie been 



r^^rted in Equality of Educational Opportunity, known 



as th(^ Coleman Report, after its principal author. 



46 






ahd a humber of serious shortcomings of the data are de 



lie sjamf 



detail lelse^ere. Any reader adventuresome enouc^i to tfHce pei: 



lajy preliminary results is urged to consult these source 
j The v^iables availab]!e for the empirical estimati 



3 



educatl 



dard df 



viations appear in 



a nurob^i 

■/- ;i"': ' - ■ 'i 

measure 
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onal production funjbtion along with their means and stan* 



rable 21. A table of the zero order 



4 ^rrel^ltion coefficients appears in an appendix, unfortunately. 



r of variables avai^llable in the Talent sainple ^e not 



id in the EEOS data 



basic equation appears in 



xn 






ind vice versa « The estimate of our 
ipable 22 • A number of coimnents are 



1 “'/ 
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’''^Colemiln, et al. Our estimations are based on the correlation 
tables and means and standJij^ deviations of each variab. Is, as 
reportef in Vol. II of Equ^^Uty o f Educational Qppprtujtti^ 



in addition to the Reporti itself, see Bowles and Levin (l968a)^^^ 
: 'Hanushok# ...and Hanuaihek and,|Kain., 
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Notes to 21 



a) Further definition of these variabltes, as well 
as the surv^ instruments on whicii theF werK^ based^ is 
available in Coleman C14) • 

b) The home environment and student attitude variables 
have been normalized to mean » d and standaipd deviation - 1 
for the national sanqple taiken ms 9 . idiole* 

c) ^ 0*99. A score of 33, 66, €^ 99 Indicates 

that the school has one, two, or all of the following types 
of labs: biology, chemistry , and physics. 

d) The verbal ability score is based cm the SAool and 
College Ability Test Scores of the Bdncatioi&al Testing 
Service. . 

I’’ ' - ' ' - - 

e) The sense of control variable is baaed on the stu- 
dent agreement or disagreement with three sl;atementss Good 
luck is more important than hard work for success; leery time 
I try to get ahead, something or somebody sfwps me; and 
People like me don*t have much of a chance to be successful 
in life. 

f) The self concept variable is based on the stadent*s^ 
responses to the following itemss How briglit do yon think 
you are in comparison with the other stndenls in your grade? 
Sometimes I feel that I just can*t learn fas|cee - disagreel; 

I would do better in school work if tfoachers didn*t go so 
fast (agree - disagree). 



appears in Table 22 • A nuniber connents are in order. 



. mm 

’A' : 



First, the parameters are roaghly consistent with onr earlier 
results. The very significsuit estimate o£ the influence of teacher 

* >r 

quality (as represented by teachers* verbal abilities acore) is 
particularly important, and not ssurpcising idien we recall that 
the teacher is by far the single laost important school input. ^ 



. The inqiortance of teacher quality has been confirmed hy 
much of the current work in l^e ctstimation of educational pro- 
duction functions. 

Given our findings from the Talent sample, the absence of a 
class size variable is surprising, the failure of a class size 
variable to appear in the equation mmy be a reflection of severe 
errors in the measurement of this variable.^® al tho ugh class size 









does. not appear to be a significant 
number of studies (for exaiiple, Banbsli^ and levin), at lea^ 

. ■ i ^ ■ ’ * ' . ’ ■ 

one author, Kiesling, found a higlily significant relatitxnship 

■' '■ ' i ■ - 

between stadents-per*-teacher and achieveoBent.^^ 



The teacher's verbal ability test consists of only 30 questions 
and is self-administered, if, as seems likely, the variance of 
the error component in this measure is large , the vestinate of 
the associated regression coefficient may be seriously dammardly 
biased. The sam reasoning, of course, applies to tiig» other 
school inputs. 

f^See Kiesling, Hanushek. In addition to these results, levin 
found that two measures of teacher qualtiy (verbal scsore type 
of college attended) were highly significant in explaining verbal 
achievement among sixth grade black students of *1 m» third socio- 
economic quartile in a large metropolitain area. 

®®See Bowles and Levin (I968b|. 

f^The negative relationship betiieen teacher-student ratio a 
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TABLE 22 

An Educational Broduction functlm 



Black Twelfth Grade Students 



Independent Variable 
(the dependent . 
v'^iable is 
_ verbal achievement) 



4. 



Beading Material in the 
home 

Mfainiber of Siblings 
(positive~few) 

parents* Educational 
Level 

Teacher * s Verbal 
Ability Score 



5. Science Lab Facilities 



Regression Coefficient 
(t in parentheses} 



Beta 



2.0931 

(2.B270) 

1.1812 

(4.2513) 

2.4213 

(4.3870) 

i.2462 

(7.1445) 

0.0505 

(2.5837) 



0.0892 



0.1288 



O. 1406 



0.2207 



0.0785 



Constant 

rI 

|X'X| 



19.4946 

(5.1938) 

0.1684 

0.6761 



nuHdbof of oft>scKva6xons 



1,000 
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Ae abaenTO. of a neasure of scbool policy, reflecting in 
pairt tshe qaallty of the interaction betwen students and teachers, 
la to be explained by the profusion of imperfect measures of this 
dimension of the input structure. Hhen we entered eleven of the 

into the ahove equation, we could 



. 



scdidol policy variables 

not accsept Ae hypothesis that all of the regression coefficients 
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if. . . 



In order to represent the 

of this set of variables, we have introduced a variable 
the extent of guidance counselling in the school as 
u Me have farther added a variable chosen to 
the general level of consunity interest in and- support 
of education, days in session. Ae resulting equation appears in 
Able .23- 

Bo A variables are highly correlated with measures of overall 




to more 



lasure of school output fbund by Welch may be a reflection 
mailer classes in rural schools and the failure to take 
of the negative influences on learning associated with a 
and comannity enviromient. CAe positive association 
icher-student ratio and tenA grade verbal scores in 
htlanta public schools estimated by j. W. Holland and 
is difficult to interpret, as the equation in which 
is reported includes a measure of per pupil expendi- 
{pins a number of insignificant variables).) Ais seems to 
hat even wiA a given level of depend iture, reduction 
sixe produces sufficiently strong effects on achievement 
than offset the associated opportunity costs. 




Ae F value leading to the rejection of the hypothesis was 
2«39 with 11 and 934 degrees of freedom. Thus the hypothesis was 
rejected at Ae 99% level of significance. 




TABLE 23 

Educational Production Function 



with School Policy and Community Support proxies 
Black Male .Twelfth Grade Students 



Independent Variable Regiressibn Coefficient Beta 

(dependent variable (t in paxrentheses ) 

is verbal achievement) 



1. 


Reading Material in the 
Home ' ’ 


1.8254 

(2.46I32]| 


: 0.0778 


2. 


Humber of Siblings 
(positive = few) 


1.7184 

[4.0405) 


0.1222 


3. 


parents ” Educational Level 


2.4083 

(4.3787) 


0. 1398 


4. 


Teacher's Verbal Ability 
Score 


1.0348 

(5.5186) 


0.1833 


5. 


Science Lab Facilities 


0.0375 

(1.8871) 


• 0.0582 


6. 


Average Time Sfient in 
Guidance 


1.4636 

(2.3364) 


i 0.0795 


7. 


Days in Session 


0.2054 

(1.9396) 


0.0588 



Constant -14.5708 

(-0.7706) 

0.1780 

|X'Xj 0.4569 

number of observations 1«000 



r 
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support for education, such as the level of teachers i salar^^^ 



and system-vide expenditure per pupil. Both are also 
associated with school policy variables such as the extent 



of 



extra curricular activities and foreign language courses, though 



•days in session* is much less closely associated with these 

53 



school variables than is the guidance counselling measure. 



. h 6 » similar are these findings W those based bn the project 
.Talent sample? In order to answer the question, cons ijper the 



two equations predicting comparable outputs : the Talent equation 






for reading comprehension (Table 12), and the EEOS equation in Table 23 
Although conqparison is difficult because the set of variables 
ia each equation is not the scune, note that the estimailed impact 
pf increasing all school inputs by a standard deviation is virtually 
identical in both equations. The impact for the Talent|| and EEOS 



equations is .35 and .32 standard deviation units in the dependent 

The estimated ^^^i^ of chan^bs in 



variable , respect ive ly 
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Of course, the days in seesibh measure may singly refllect urban- 
rural differences, as we have considerable evidence tha'|b rural 
Schools are open for f^^ days per year. (See Coleman} As a 
test of this hypothesis , we added a variable measuring the size 
of t^ Senior class to the equation^ This new variable j was insig- 
nlfiCailt, and, although its introduction lowered the esl|imated 
regression coefficient for in sessibn* by about ten percent, 
the latter variable was still significantly dif ferent fr|om zero at 
the 95% significance level. The remainder of the equati|pn was 
altered /only slightly# /"v 
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This figure is the sum of the beta coefficients for 
inputs; the days in session variSble is regarded as a cqmnunity 
variable* not a school 



er|c 









I 
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social baclcgiraund is roughly similar in both equations (,62 standard 
deviation units for the ^Talent data and ,40 for the EEOS), as is 



o ' 

me 



the portion of, variance explained in both equations (0,2444 for 



Talent and 0, 1780 for the EEOS ) . 



some fur fiber support for the stability of these find ihgs 



is contained in Tables 24 arid 25., Which present equations esti- 
mated from the EEOS data for samples of Southern and Northern 
students sepatately , (These sairiples are different from the 



patibhal EEOS ; sample used thus far, ) Taking into account 



the numerous ^Shortcomings of the data and methods used, the 



similarity ini; results is striking* Althou^ the above evidence 



is much too w^sak to demonstrate the generality of the project 
Talent re suit ii# none of the available information is seriously 
inconsistent With the general qualitative outlines of the functions 



based on Ta lei^t data , 



iii 
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T^ble 24 



Estimated Regressions for Samples of Northern 12th Grade 

Students 



r. 



Independent Variable (dependent 
variable is verbal achieye- 

yl ’/. ..rnent) 


Regression Coefficieni: 
(t in parentheses) 


Beta 


1.^ Reading Material in the 

X. . .y ,^;.y;^ : 


■ ; ■ X.279 

(1.6013) 


.052 


2* Niunber of Sib^^^ 

/’ (positive « few) 


: 1.660 


.116 


3 . Parents • Educational Level 


" 2.65S " 

(4.626) 


.151 


4. Family Stability 


' ;’.''899 : ■ ■ ■'■V.': 

(1.675)' : 


.051 


5. Teacher's Verbal Ability 


.721 ■ ■ 

v':. . ' : (3.193) ' 


.097 




.059 

(2.137) 


.067 


7 « ; Days in Session 


.189 V ' 'V : 

(1.971) 


.062 






Constant 

. - - 



^ . ! 

11-0.1462) 






number of 



observations 1> 



I • 
[■. 



;>■■■■: I 







■ \ 



•• 'i 

■•.;■. > ■: 






‘ I 



mu 



HBI 



















•T, % • 



3 '! 



- ‘,r r<i: ‘VI «» >•■• -s:-*. {■ ■ It V".~’ ' 'r f V: 

, . '■. ■•■‘■i'! .• ••, ; 



!■' 
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table 26 



Estimated 



for Samples of Southern 12th Grade 
Students 



Indepen^nt variable (de^ndent 
variable is verbal achieve* 

; -ment)'^ :■ 



Regression Coefficient Beta 
(t in parentheses) 






1 . 


Reading Material in the 
Home- 1' . 


; ■ ■1.84XV . 

(2.629) 


.083 




■/ ;-'il 

Number i 
llK>s4^ 


of Siblings 
tive •» few) 


'I-.794 ‘ y' y 

(4.438 


.135 




Rarents 


I* Educational Level 


yy ^-^vy^- ^;. X 85-' ■ 

(4.181) 


.132 




Family Stability 


y.:y - yi .823 

(1.858) 


.053 




Teacher' 

Score 


|*s Verbal Ability 


1.097 

(6.593) 


.210 




Science 


; Lab Facilities 


.027 

(1.724) 


.052 




Average 


1 t. . ' ; . ‘ ■ 

1 Time Spent in Guidance 


2.017 

(3.266) 


.102 




% 









X. SPBCIPICATICii BIAS 
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Thus far we have been working with a model i 
explicxt account is taken of stilident endowments a 



a which no 

I 

b the beginning 



of school. The biases in our esiitimates resulting fr^^m this 

' ■■ ■ I' ■ " ; . ' . ■ 

exclusion are suggested by the iiiollowing exercise 



attempted to explain a similar achievement score 



We have 



our set of •explanatory variables. The resulting ( 
the calculation of the s;pecificaibion bias appear ;i 



it grade one by 
M|uation and 



.n Table 26. 



At the first grad€5 level r the school input variabj 



.c»s were never 



levels), 
the technique 



significantly different from zero (at conventional 

I , ' 

Given the crudenesis of both the measures and 

the part xcular numerical, estimates are subject to Icons iderable 

■ ' ' . ' ^ ■ -■ ■ . • ' '!' 

error. Nonetheless, as ex;pected,, the apparent influence of 

social class on school learning is drastically red 
the significance of school inpute is not affected, 
as long, as we use an additive linear model with net 
effects and plausibly assume no effect of school i 
initial scores, there can be no estimated bias of 
inputs . Then our result is hardly surprising. 



need, while 
^Of course, 
interaction 

n . 

liiputs on 
the school 



®^The analytsis in this section is restricted to th 
as the project Talent data at my disposal do not ii 
one scores or the relevant student attitude measure 



i BEOS data, 
iclude grade 









Table 26 

Coritection For Specification -iias Due to Omitted initial 
Endowments in the Educational production Function. 

Slack twelfth Graders. 






Regression Co< 
efficient at 
grade twelve 



Home En^Hronment : 

Reading;! Material, in 
the 

Mumber cjf Siblings 
(posiiiive = few) 

- - ■. ' ' I' ' ' 

Fanents ' Educational 
Level 



School E nvironment : 

ii 

Teacher's Verbal 
Ability Score 



Science Lab Facil- 
ities 

■ , '/-fr ■ : ■ : 

Average itime Spent 

in 



in $ession 



at grade 



one* 



Corrected Re- 
gression Coef’ 
ficientst> 



-I 



( 1 ) 


(2) 


(3) 


1.8254 


• 348 
(1.97) 


1.1969 


1.7184 




1.7184 


2.4083 


.884 


.812 


i - 


(5.85) 


, 


1.19348 


m 


1.041 


.(9375 


— 


.0375 

S' 


1.^1636 


— 


1.4636 


.02054 


— 


•02054 



a) t raivios are in parenthcises • The coefficient of determination 
for the equation was .05^ 

b) I Colunin (3) = Column (1) - Column (2) x.b, where b, 

the iegression coefficient of A- in equation (7), is aSiOmed 
to be . or 1.806 . 




Recall that equation (2) represented our reduced form. 

Yet a complete specification of the learning envieonment must 
plausibly include student motivations., in our case these 
attitudes are represented by tvo measures, student self-concept, 
and sbudent. sense of control over environment When these are 
added to the basic equation, we arrive at the equation presented 
in Table 27." 

It is iforth noting that the structural parameters related 
to the school inputs change very little, suggesting that, in this 
case, the simultaneous equation bias is relatively small. The 
attitude variables are powerfully related to achievement — the 

proportion of variance explained is almost doubled by their inclu- 
sion. 



S^See T Ale 21 for the measurement of these variables. 
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TABLE 27 

Educa^icnal production Function wltli Student Attitudes Measured 

Black Male Twelfth Grade Students 



Independent Variable 
---(dependent variables is 
verbal achievement) 


Regressioii Coeffi- 
cient |t in paren- 
theses) 


Beta 


1. Reading Material in the 
Home , 


0.5686 

(0.8243) 


0.0242 


2.. Humber of Siblings 
(positive =‘ few) 


1.5091 

(3.8459) 


0.1073 


3. Parents' Educatitmal Level 1.8527 

{3.63901 


0.1075 


4. Science Lab Facilities 


0.0355 

(1.9392) 


0.0552 


5. bays in Session 


0.1821 

(1.8653) 


0.0521 


6 . Teacher " s Verbal Ability 
Score . 


1.1069 

(6.3977) 


0.1960 


- ■ ' * i ■ ■ - 

7i Average Time Spent in 

Guidance 


1.7673 

(3.0523) 


0.0960 


8. Student's Control of 
Environaent 


4.4418 

(8.3100) 


0.2353 


9. Student's Self-Concept 


4.2767 

(7.4531) 


0.2111 



Constant —12.2473 

(-0.7020) 

0.3031 

[X*x| 0.3870 

number of obs:?.r vat ions 1,000 



XI* CONCLUSION: THE DEFECTS OF SCHDOLIHG OH ACUIEVEMEHT 

The imperfect measurement^ the limited eiqposiire to the 
educational environment, and our fundamental Ignorance alKint hcnr 
children learn estad>lish the presunptlon that the estimated 
effect of different schools id.ll be gulte limited. Nonetheless, 
our estimated, equations suggest that the difference in a<diieve- 
ment between students in schools with inputs at levels one 
standard deviation below the mean for our sample coapared with 
students in schools one standard deviation above the mean ranges 
from .64 to 1.5974 standard deviations on our acliievenent scale.®"^ 




. Given the limited nature of the sample, and the inadequate 
opportunity to explore the available data, i will refrain from 
generalizing from these initial, encouraging results. Nhat we 
have uncovered so far, however, suggests that oar emptesis on the 
school in the study of human ci^ital formation and income dis- 
tribution is not misplaced . It is certainly the most important 
learning environment directly under social control, ite have 
identified a number of school inputs idiich do seem to affect 
learning. . 



For the purposes of these calculations, length of schcMl year 
is considered a community variable, not a school inputs. 



X] 
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POSTSCRIPTS INITIAL RESULTS GCXICERNING THE EFP^BCT OF SCBOOL 
QUALliTY UPON EARNINGS 

in the analysis of the effect of schooling on econoadc 
3wth and the distribution of income, the relationship between 

ication and earnings plays a central role. Studies of this 

'I , ■■ " ■■ ' . ^ 

tionship have ordinarily measured education by years of 



';re|lati6nshi 

ac^lloolina.^ 



of 



on 



CO 

tooling. . Recently a number of authors have investigated the 
at ionship between scholastic achievement and warnings 

Here;! adopt a different approach, namely the investigation 
the relationship between the level and quality of school 
uts, oh the one .hand, - and earnings|^^ in post-school employment, 
the Other. importance of the p^ticular 
ensions of school inputs, I deal with students all of whom 
ehed the twelfth grade. 



Oar data on earnings are for the first job after leaving .school, 
eported by the student, and are undoubtedly considerably in error, 
ence hf bias is clear from the fact that the reported monthly 
eaxjiiings of respondents in this sample are 1.67 times the earnings 
he analogous group as reported in the U*S. census, iftsis the 
ings below should be regarded as esq^loratory hypotheses for fiirther 



inoch. 



59 



' 1 • V'>‘ M ' 

Hl\ liss, and Weisbrod, Hansen and Scanlon, 
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similar approach has been used by Welch. The results 
based entirely on the Project Talent data. 



stLiidy* pcelivinaxy analysis of the ^ata on the Project sample 

of hlac^ in the u.S. Office of Education regions 1, 2 , and 3, and 

• ^ 

idio mere seniors in I960 resulted in the equation which appears 
in ^sthle 28. HOte the followings 

First • the relationship between social baclcground and 
earnings in the first job is very weako (See Table 29.) 

Second, t her e is no statistically significant relation 
between the neasuces of scholastic achievement and earnings. The 
t-statistics for each test score idien added to the. above equation 
appears in Table 30.^^ - 

Third, sone school inputs are significantly related to 
^amings,^ even for individuals with the same number of years of 



The above findings suggest an interesting and important 
prcq^ition for further study. They suggest first that schools 
l^forsi more than a selc^ion function and produce a unique 
effect upon earnings, but, secx>nd, that this effect is not primarily 



Annual earnings of the Project Talent respondents were calculated 
using ten months of employment per year. The discrepancy probably 
reflects both erroneous responses and the unrepresentative nature 
of the Talent sample. 

62 

Knsen« Weisbrod, and Scanlon found a statistically significant 
relationship between the AFQT and earnings in a group of low- 
a^ieyi^ Northern blacks. Although their findings heighten my 
slceptlcism concerning the above estimation, it should be kept in 
mind that the two population groups are not at all comparable. 

The mean years of schooling in the Hansen, Weisbrod, and Scanlon 
sa^le is nine years and measured achievement is around the third 
grade equivalent level. 




Table 28 

An Bducational production Function: Dependent 

variable is Starting Monthly salary 
.. Blacdcs Mho wcnre Tiielfth Grade Students in 1960 



A -■■■ 






independent variable 


Regression C<^f ficient 
(t in parentheses ) 


Beta 


1. Educational innovation 


31.62 


.1845 




(2.5988) 




2. percent in College prep 


, * . ' ‘ X 

. 1.50 


.1827 




(2.5921) ' 


.-a 


3 . Part-Time Teachers/Total 


-345.04 


-.1461 


Classes in School 


(-1.9349) 




4. Class Size, Science & Math 


-6.09 

(-1.6715) 


-.1269 



Constant 57.9877 

(0.2783) 

R^c .0612 

|x*xl . .7200 

nanber of observations, see Table 9 



Table 29 



Relation of Socio-EconomiG status of parents to Eaxrnings 

t-Statistics for SES Variables 



Socio-Economic Status 
variable’ 


t-Statistic® 


Affect of Higher 
Status on Earnings 


1 • Father ' s Occupation 


• 0804 




2* Mother's Occupation 


1.1442 


■ ■ ■ 


3* Father's Education 


1.3793 




4 • Mother * s Education 


.5989 




5. Own Room, Desk, Type- 


;0927 




writer 






6. Appi^liances^ 


.3939 


+ ’■ ■' ■ 


7. TV, Telephone, Radio, 


.4220 




Phonograph 






8* with Whom Living 


1.3147 





^he t-statistics reported here are for each SES variable 
when added to the equation in Table 28, 












'' ’ i ' ^ ' > .**«r 



?T\. 



Af**% 



iiiSSSSi^Si^liS^^ 

Relation of Scholaet ic 
tVsfatiatics f 
Dependent var iatlXe is 
Rlacke Who Wero welf th 



.')■ \\ .'■■C' 
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Jj^chievement to warnings 
3>r , Test Scores: : -■■■' 
i|artii^r salary 

Grade students in 1960 



■ i'. 



' '*! ■ ■ ’ 
■•I'; 



,1 ;. 



t-' ■.-■ 
[ ' •* 
i ■ *:: 



■ f . 






. WJJI ...... / . 


, Test 


t-sibatistic? 


Affect of Higher 
Achievement On 

.:„.-.j....Earnings ' 








Reading cbrnprehension 


. 1151 ' _ 




:iiathei^a^^ 


. 1092 




General Acadeittic Ability 


'.1325, 




®The t-statistics reported here are for each 
when added to the eguiatioh in Table 28. 


■■’ - ' . ; : //y-- ■ ■ ■ ' ' ' ■ 

: ■ .■_ ;' _•■• •, .’ ‘ -‘ 'w . ■ '■ ■■■ , *i ■ ■ ‘ ' 

best': score: 




. V ' ' ' 






■V h' -V ■" V!'* -V,' : /'V •. 
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o . 



o 

me 



£531 















-97- 



'. .4f'- ?f-'> 






conveyed through cognitive development (as measured in scholastic 
achievement scores ) / but through other effects of schooling oh 
earnings 
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If the proposition is correct, we must renew the search for 



ade<^ete economically relevant school output and 



aubstantially revise our. view of the role of education in the 



The proposition will be tested with other 
b of Project Talent data in my forthcoming work. The above 

, Jv : res by themselves are no more than suggestive. 


















Strong case for the second part of this propos it ion is made 
by Heirbert Gintis in his, ''Toward a Method in the Economics of 
Education --The Educational production Function, " unpublished 
minieo. Harvard University, 1969. 
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Table 32, 

Zero Order correlation Among Vaiiriablee 



variable number 1 

variable ream 


2 


3 


4 


5 6 


7 


1 


Reading Comprehension 1.000 


. 5170 


.8855 


.027CI 


.2955 .2016 


.2157 


2 


Mathematics 


1.000 


.7301 


. 0414 


.2097 .1348 


.1738 


3 


General Academic Ability 




1.000 


.0368 


/ 

. 2936 . 2096 


.2212 


4 


salary 






l.OGO 


-.0105 -.0644 


.1217 


5 


Father's Education 








1.000 .2701 


.4227 


6 


Mother's Education 








1.000 


.2992 



7 Father's Occupation 1.000 



8 Mother's Occupation 

9 Own Room Desk Typewriter 

10 Appliances 

11 TV, Telephone, Radio, 
Phonograph 

12 with Whom Living 

14 Class Size, Sci & Math 

15 Senior Class Size 

16 Educational innovation. 

17 Starting Salary, Male, 

B. A • 

18 Teachers Fully Certified 

19 Percent College Prep 

2Q Teacher's Starting salary, 

21 Tracking 

22 Percentage Negro 



I 

i 

I 



er|c 






- 100 - 



TD- 



TT 



TZ 



TT 



“ I5 " 



2515 



2 .1311 

3 .2258 

4 J .0525 

,5 .2830 

6 .3354 

■ . .'.'•5524 
3 1.000 
; 9 , . ' '■ .• 

10 
11 
12 

14 

15 - 

16 
IT 

' ‘ ■ ' ' ") i ' ■ 

18 

19 

• : ■ ’?o 

21 

22 



.1212 .1334 

. 0928 . . 1456 . 

. 1396 . 1634 

-.0270 .0455 

.1230 .0852 

. 144t .0887 

. 1932 . 1647 

.0207 .1888 

1.000 .3215 

1.000 



.3704 -.0659 

.3418 -.1229 

.4573 -.1331 

.0020 .0625 

.1621 -.2212 
.0973 -.0059 

.1677 -.1947 

.2286 -.0716 

.2934 -.1601 

.4738 -.1325 

l.OdO -.1155 

1.000 



-.1970 


-.1032 


-. 1358 


;0166 


-.2136 


-.0262 


-.1154 


.0524 

f 


.0045 


.0511 


-.0546 


-.0521 


-.0458 


.0607 


-.0322 


.0325 


.0677 


.0919 


-.0816 


-.0976 


.0107 


.0791 


.0157 


-.1332 


1.00 


.4522 



1.00 ; 

i_. 

■. ■ •!*' 
■ !■ 

• ■ [■ 









. V ' ■ 












— J . 

1 ■' 

'4 




T ' 


*■ 




1 ^ 


- 101 - 






































: ./■■,■■ 


>; ■ . *♦ 

16 


17 


18 


19 


20 


21 


22 


: i n 

1 

' ‘ '■ ■• ' • • 


-.0085 


.0499 


-. 1330 


.1323 


.1826 


-.2275 


-.1088 




.0979 


.0093 


-.0477 


.1529 


.2186 


-.1749 


-.1976 


: ". ' '■ '."k ■. 

.-3 . ■ 


.0551 


.0239 


-.1007 


.2166 


.2190 


-.2538 


-.1660 

.1207 




.1761 


.0348 


.0574 


.1338 


.0003 


-.0523 


■ is '- 
; ■■■ 


-.0686 


.0508 


-.1872 


.1480 


.0398 


.0171 


-.1412 




-.0367 


-.0412 


-.0115 


.1036 


.1304 


.0006 


.0357 




.0475 


-.0208 


-.1415 


.1583 


.0930 


-.0638 


.0980 1 




.0252 


.0650 


-.0783 


.0289 


.0477 


-.0413 


. 1220 ' - ' ; 




-.0820 


.1412 


.0899 


.0581 


.1256 


.0090 


-.1295 


I 10 

* ■-■ .■•..-■■.■ - • 
t .■.'■ ■■■ ■ ' 


.0138 


.0298 


.0090 


.0504 


.0394 


-.0130 


.0436 , 


; ii , ^ ; 11 ^ 


.0216 


.1404 


.0049 


.1356 


.0839 


-.1200 


-.0501 


: ■:.®2 


-.0134 


-.0255 


.0687 


-. 1143 


-.0843 


-.0673 


• 1367 


i v'w 


-.1256 


.1550 


.2373 


-.1405 


-.1178 


.4300 


-.0366 




.2926 


.1370 


.1190 


.1829 


-.2763 


.3847 


-.3215 


iv ^ v -;: ii 6 ;: 


1.000 


.0253 


.1120 


-.1164 


-.1334 


.0599 


.3012 






1.000 

II 


.0647 
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